Jul 18 02:40:08.095428 ip-10-0-138-139 systemd[1]: Starting Kubernetes Kubelet... Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.492597 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497090 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497100 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497106 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497110 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497114 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497117 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497120 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 02:40:08.516088 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497124 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497127 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497129 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497132 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497135 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497142 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497145 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497148 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497151 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497153 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497156 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497159 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497161 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497164 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497167 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497169 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497172 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497174 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497177 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497180 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 02:40:08.519118 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497182 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497185 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497188 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497192 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497196 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497199 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497202 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497206 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497208 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497211 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497214 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497216 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497219 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497222 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497225 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497227 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497229 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497239 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497242 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 02:40:08.533140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497245 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497247 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497250 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497252 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497255 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497257 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497260 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497262 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497266 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497270 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497273 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497276 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497279 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497281 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497285 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497288 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497291 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497293 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497296 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497298 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 02:40:08.534946 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497301 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497304 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497306 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497309 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497311 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497314 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497317 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497320 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497323 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497325 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497328 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497339 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497342 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497344 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497347 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497349 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497352 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497354 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497357 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 02:40:08.537402 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497360 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497362 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497367 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497370 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497372 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497375 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497377 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497380 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497382 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497385 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497387 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497390 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497392 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.497395 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498098 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498104 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498106 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498109 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498112 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 02:40:08.546074 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498114 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498118 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498120 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498123 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498126 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498129 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498132 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498134 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498137 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498140 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498142 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498145 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498147 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498150 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498152 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498157 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498160 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498162 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498165 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498167 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 02:40:08.546890 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498170 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498172 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498175 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498177 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498180 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498183 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498186 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498189 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498191 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498193 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498196 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498199 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498201 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498206 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498209 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498212 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498215 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498218 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498221 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 02:40:08.552030 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498223 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498226 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498229 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498231 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498234 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498236 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498239 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498241 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498246 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498248 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498251 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498254 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498257 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498260 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498262 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498265 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498267 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498270 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498273 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498275 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 02:40:08.552567 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498278 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498281 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498283 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498286 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498288 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498293 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498297 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498300 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498306 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498309 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498311 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498314 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498317 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498320 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498322 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498325 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498328 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498330 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498333 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 02:40:08.553465 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498335 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498339 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498341 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498344 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498347 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498350 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498352 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498356 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498359 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498363 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498365 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498368 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498371 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498373 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498376 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498379 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498738 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498742 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498745 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498748 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 02:40:08.554324 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498751 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498754 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498756 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498759 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498761 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498764 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498768 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498772 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498774 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498777 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498780 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498783 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498785 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498789 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498791 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498794 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498797 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498799 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498802 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 02:40:08.554868 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498805 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498808 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498811 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498813 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498816 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498818 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498820 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498823 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498826 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498828 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498831 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498833 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498836 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498839 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498841 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498843 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498846 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498849 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498852 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498854 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 02:40:08.555417 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498857 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498860 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498862 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498865 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498867 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498869 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498873 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498875 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498878 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498880 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498883 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498885 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498889 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498891 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498894 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498896 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498899 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498901 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498904 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498907 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 02:40:08.555928 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498909 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498911 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498914 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498916 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498919 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498921 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498924 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498926 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498928 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498931 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498933 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498936 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498938 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498941 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498945 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498947 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498950 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498952 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498955 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498958 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498961 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 02:40:08.556508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498963 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498978 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498982 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498987 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498991 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498994 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.498997 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499000 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499003 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499005 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499008 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499010 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499013 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499016 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499018 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499086 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499096 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499102 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499107 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499111 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 02:40:08.557096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499115 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499119 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499122 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499126 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499128 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499132 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499135 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499138 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499141 2560 flags.go:64] FLAG: --cgroup-root="" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499145 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499148 2560 flags.go:64] FLAG: --client-ca-file="" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499151 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499154 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499157 2560 flags.go:64] FLAG: --cluster-domain="" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499160 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499164 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499167 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499170 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499174 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499177 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499180 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499183 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499186 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499189 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 02:40:08.557685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499191 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499194 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499197 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499201 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499204 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499206 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499209 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499212 2560 flags.go:64] FLAG: --enable-server="true" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499215 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499219 2560 flags.go:64] FLAG: --event-burst="100" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499222 2560 flags.go:64] FLAG: --event-qps="50" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499225 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499228 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499231 2560 flags.go:64] FLAG: --eviction-hard="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499235 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499237 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499240 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499244 2560 flags.go:64] FLAG: --eviction-soft="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499247 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499249 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499252 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499255 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499257 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499261 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499264 2560 flags.go:64] FLAG: --feature-gates="" Jul 18 02:40:08.632926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499268 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 02:40:08.576936 ip-10-0-138-139 systemd[1]: Started Kubernetes Kubelet. Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499270 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499273 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499276 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499279 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499282 2560 flags.go:64] FLAG: --help="false" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499285 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499288 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499291 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499293 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499296 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499299 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499302 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499305 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499308 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499311 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499314 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499316 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499319 2560 flags.go:64] FLAG: --kube-reserved="" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499322 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499325 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499328 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499331 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499333 2560 flags.go:64] FLAG: --lock-file="" Jul 18 02:40:08.633983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499337 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499340 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499343 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499348 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499351 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499353 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499357 2560 flags.go:64] FLAG: --logging-format="text" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499360 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499363 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499366 2560 flags.go:64] FLAG: --manifest-url="" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499369 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499373 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499376 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499380 2560 flags.go:64] FLAG: --max-pods="110" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499382 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499385 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499388 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499391 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499394 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499396 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499399 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499406 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499408 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499411 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 02:40:08.634961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499414 2560 flags.go:64] FLAG: --pod-cidr="" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499417 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499422 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499425 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499428 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499431 2560 flags.go:64] FLAG: --port="10250" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499434 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499437 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab3410f151fa5be1" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499441 2560 flags.go:64] FLAG: --qos-reserved="" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499444 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499447 2560 flags.go:64] FLAG: --register-node="true" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499450 2560 flags.go:64] FLAG: --register-with-taints="" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499453 2560 flags.go:64] FLAG: --registry-burst="10" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499456 2560 flags.go:64] FLAG: --registry-qps="5" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499458 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499462 2560 flags.go:64] FLAG: --reserved-memory="" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499466 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499469 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499472 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499475 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499477 2560 flags.go:64] FLAG: --runonce="false" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499480 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499483 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499486 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499489 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 02:40:08.635754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499492 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499494 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499498 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499500 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499503 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499506 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499508 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499511 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499514 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499517 2560 flags.go:64] FLAG: --system-cgroups="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499519 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499524 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499527 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499530 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499534 2560 flags.go:64] FLAG: --tls-min-version="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499538 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499541 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499543 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499546 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499549 2560 flags.go:64] FLAG: --v="2" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499553 2560 flags.go:64] FLAG: --version="false" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499556 2560 flags.go:64] FLAG: --vmodule="" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499565 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.499568 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 02:40:08.636531 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499672 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499676 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499679 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499682 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499685 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499688 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499691 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499694 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499697 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499700 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499732 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499736 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499740 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499743 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499746 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499748 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499751 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499754 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499756 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499759 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 02:40:08.637267 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499761 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499764 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499767 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499771 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499774 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499777 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499780 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499782 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499784 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499787 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499791 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499793 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499796 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499798 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499801 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499804 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499808 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499811 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499814 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499817 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 02:40:08.638027 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499820 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499823 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499825 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499828 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499831 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499833 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499836 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499838 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499841 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499843 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499846 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499849 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499851 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499854 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499856 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499859 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499862 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499865 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499868 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499870 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 02:40:08.638716 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499873 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499875 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499880 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499884 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499886 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499889 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499891 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499894 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499896 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499899 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499901 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499904 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499906 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499909 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499913 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499916 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499919 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499922 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499925 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 02:40:08.639315 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499928 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499930 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499933 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499935 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499938 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499940 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499943 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499945 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499949 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499951 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499954 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499957 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499959 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499961 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499964 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499981 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499984 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499986 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499989 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 02:40:08.639895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.499991 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.500519 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508084 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508099 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508117 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508123 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508186 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508190 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508193 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508197 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508200 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508202 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508205 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508208 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 02:40:08.640886 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508212 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508216 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508219 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508222 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508224 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508227 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508230 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508234 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508237 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508240 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508242 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508245 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508247 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508257 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508260 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508264 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508266 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508269 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508272 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 02:40:08.641394 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508275 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508279 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508282 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508285 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508288 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508290 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508293 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508295 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508298 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508301 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508304 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508306 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508308 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508311 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508313 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508316 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508318 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508321 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508323 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508326 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 02:40:08.779457 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508332 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508335 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508337 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508339 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508342 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508345 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508347 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508350 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508353 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508356 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508359 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508361 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508364 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508366 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508369 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508371 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508374 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508377 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508379 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508382 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 02:40:08.780553 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508384 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508387 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508390 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508392 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508394 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508397 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508399 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508402 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508404 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508407 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508410 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508412 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508416 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508420 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508424 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508426 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508429 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508431 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508434 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 02:40:08.781426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508437 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508441 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508444 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508446 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508449 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508451 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508454 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508456 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508459 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508461 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508464 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508467 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508469 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508474 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508596 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 02:40:08.782459 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508601 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508604 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508607 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508610 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508613 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508616 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508618 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508621 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508623 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508626 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508630 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508633 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508636 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508638 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508641 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508644 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508646 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508648 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508651 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 02:40:08.783109 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508655 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508657 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508660 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508663 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508665 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508668 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508670 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508673 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508676 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508678 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508681 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508683 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508686 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508688 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508691 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508695 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508698 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508701 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508703 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 02:40:08.783992 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508707 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508711 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508714 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508716 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508720 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508723 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508726 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508728 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508731 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508735 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508738 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508741 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508744 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508748 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508750 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508753 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508755 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508758 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508761 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508763 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 02:40:08.784819 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508766 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508768 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508771 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508773 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508776 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508778 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508781 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508783 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508786 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508788 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508790 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508793 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508796 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508798 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508801 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508803 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508807 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508809 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508811 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508814 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 02:40:08.785880 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508817 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508819 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508822 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508824 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508827 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508830 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508833 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508835 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508837 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508840 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508842 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508845 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508847 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508850 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508853 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508855 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508858 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508860 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508863 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 02:40:08.786747 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:08.508865 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508871 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.508987 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.511809 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.513132 2560 server.go:1019] "Starting client certificate rotation" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.513228 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.513272 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.516065 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.517958 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.525702 2560 log.go:25] "Validated CRI v1 runtime API" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.530441 2560 log.go:25] "Validated CRI v1 image API" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.534331 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 02:40:08.787604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.539127 2560 fs.go:135] Filesystem UUIDs: map[0b232d80-34e3-4402-b9cc-06e5c5b25b32:/dev/nvme0n1p3 53eb278f-b97a-4d07-8f8c-22385a339fe1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 18 02:40:08.788246 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.539173 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.556998 2560 manager.go:216] Machine: {Timestamp:2026-07-18 02:40:08.555750282 +0000 UTC m=+0.344524042 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096885 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f7fa7ca221268f520df0e0db20e75 SystemUUID:ec2f7fa7-ca22-1268-f520-df0e0db20e75 BootID:0193c19e-59b5-472c-a951-ed45ea0b88d5 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0d:d0:0c:d7:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0d:d0:0c:d7:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:6c:e9:55:fe:4e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.557095 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.557188 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.559520 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.560670 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.560698 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-139.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 02:40:08.788456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.561416 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.561426 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.561434 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.561920 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qr8pt" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.562170 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.563337 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.565931 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566045 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566072 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566081 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566091 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566099 2560 kubelet.go:398] "Adding apiserver pod source" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566108 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566364 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.566376 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.568174 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.569573 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.569600 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570101 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570113 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570119 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570124 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570132 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570138 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570241 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570248 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570254 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570260 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570272 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.570280 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.571514 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.571521 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.574291 2560 server.go:1268] "Started kubelet" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.574570 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qr8pt" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.574877 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.574914 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.575337 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.577191 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.577290 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.579805 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-139.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 02:40:08.788888 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.579853 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-139.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.579920 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.583256 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.583607 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.585462 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.585993 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.586001 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.586108 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.586164 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.586168 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.586525 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.586907 2560 factory.go:55] Registering systemd factory Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.586923 2560 factory.go:223] Registration of the systemd container factory successfully Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587120 2560 factory.go:153] Registering CRI-O factory Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587131 2560 factory.go:223] Registration of the crio container factory successfully Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587168 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587185 2560 factory.go:103] Registering Raw factory Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587194 2560 manager.go:1196] Started watching for new ooms in manager Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587485 2560 manager.go:349] Starting recovery of all containers Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.587992 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.596361 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-139.ec2.internal\" not found" node="ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.605733 2560 manager.go:354] Recovery completed Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.609459 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611234 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611267 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611283 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611737 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611748 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611753 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.611763 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.612405 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.612429 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.612443 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.613065 2560 policy_none.go:49] "None policy: Start" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.613083 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.613095 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.614038 2560 policy_none.go:47] "Start" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.653597 2560 manager.go:337] "Starting Device Plugin manager" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.653636 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.653645 2560 server.go:87] "Starting device plugin registration server" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.653844 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.653857 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 02:40:08.790604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.653930 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.654026 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.654036 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.654480 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.654531 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.749593 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.750689 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.750724 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.750740 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.750768 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.752818 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.754147 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.754872 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.754896 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.754905 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.754924 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.763528 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.763544 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-139.ec2.internal\": node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:08.792385 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.776752 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.851297 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal"] Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.851461 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.852639 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.852663 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.852672 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.853677 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.853773 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.853796 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.854304 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.854305 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.854328 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.854339 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.854345 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.854355 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.855543 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.855562 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.856163 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientMemory" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.856185 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.856197 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeHasSufficientPID" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.876849 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.888019 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1439b4208df4c574b43f6dbf41e494ee-config\") pod \"kube-apiserver-proxy-ip-10-0-138-139.ec2.internal\" (UID: \"1439b4208df4c574b43f6dbf41e494ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.888040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ecb42b15a64d1776d35f748981073b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal\" (UID: \"9ecb42b15a64d1776d35f748981073b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.888061 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ecb42b15a64d1776d35f748981073b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal\" (UID: \"9ecb42b15a64d1776d35f748981073b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.890428 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-139.ec2.internal\" not found" node="ip-10-0-138-139.ec2.internal" Jul 18 02:40:08.900864 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.894268 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-139.ec2.internal\" not found" node="ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:08.977641 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.988882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1439b4208df4c574b43f6dbf41e494ee-config\") pod \"kube-apiserver-proxy-ip-10-0-138-139.ec2.internal\" (UID: \"1439b4208df4c574b43f6dbf41e494ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.988912 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ecb42b15a64d1776d35f748981073b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal\" (UID: \"9ecb42b15a64d1776d35f748981073b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.988928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ecb42b15a64d1776d35f748981073b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal\" (UID: \"9ecb42b15a64d1776d35f748981073b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.988982 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1439b4208df4c574b43f6dbf41e494ee-config\") pod \"kube-apiserver-proxy-ip-10-0-138-139.ec2.internal\" (UID: \"1439b4208df4c574b43f6dbf41e494ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.988995 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ecb42b15a64d1776d35f748981073b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal\" (UID: \"9ecb42b15a64d1776d35f748981073b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.048430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:08.989004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ecb42b15a64d1776d35f748981073b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal\" (UID: \"9ecb42b15a64d1776d35f748981073b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.191528 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:09.078181 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.191528 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:09.178496 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.197159 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.192818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.198415 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.197312 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.299358 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:09.278697 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.407598 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:09.378988 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.525797 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:09.479345 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.525797 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.512680 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 02:40:09.525797 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.512780 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 02:40:09.525797 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.512809 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 02:40:09.647919 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.579064 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 02:35:08 +0000 UTC" deadline="2028-04-08 10:37:21.68605336 +0000 UTC" Jul 18 02:40:09.647919 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.579093 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15127h57m12.106963056s" Jul 18 02:40:09.647919 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:09.580141 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-139.ec2.internal\" not found" Jul 18 02:40:09.647919 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.598541 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.686640 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.699529 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.701356 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.710909 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.718082 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.752919 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" event={"ID":"1439b4208df4c574b43f6dbf41e494ee","Type":"ContainerStarted","Data":"450aa40812a7288df417145ce42ed6ff26cbd4dda3f86fb84808daf1d92a25f9"} Jul 18 02:40:09.764893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.753672 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" event={"ID":"9ecb42b15a64d1776d35f748981073b4","Type":"ContainerStarted","Data":"b236c707a376017b513e538055de17cd89fa0c640bede75e6f8c64ff5a1b822d"} Jul 18 02:40:09.868404 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:09.835342 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 02:40:10.502231 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.502069 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 02:40:10.567160 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.567134 2560 apiserver.go:52] "Watching apiserver" Jul 18 02:40:10.574427 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.574187 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 02:40:10.574747 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.574645 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-4grnx","kube-system/konnectivity-agent-mqgz7","kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wqhjw","openshift-dns/node-resolver-7zpc4","openshift-image-registry/node-ca-9b5gs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal","openshift-network-operator/iptables-alerter-rq5t9","kube-system/global-pull-secret-syncer-nswl4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c","openshift-multus/multus-5sj4d","openshift-multus/multus-additional-cni-plugins-x5g6r","openshift-multus/network-metrics-daemon-jzqvv","openshift-network-diagnostics/network-check-target-ltr2b"] Jul 18 02:40:10.577277 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.576306 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.579277 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.579259 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:10.579359 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.579330 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:10.580025 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.579754 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.580025 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.579842 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 02:40:10.580155 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.580031 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.582221 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.580265 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c9f2v\"" Jul 18 02:40:10.582221 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.581995 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:10.582221 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.582057 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:10.583411 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.583042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.585183 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.585165 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.585613 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.585342 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.585613 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.585439 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6ngvk\"" Jul 18 02:40:10.585613 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.585554 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 02:40:10.586659 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.586037 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.587561 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.587542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.588247 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.588074 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.588322 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.588297 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.588593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.588439 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6hwpp\"" Jul 18 02:40:10.588831 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.588736 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.588831 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.588803 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:10.590381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.589977 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.590381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.590127 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 02:40:10.590381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.590198 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pd8fw\"" Jul 18 02:40:10.590381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.590291 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.590640 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.590623 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.592090 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.591915 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.592647 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.592629 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.592722 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.592638 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6qcc8\"" Jul 18 02:40:10.593261 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.593089 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 02:40:10.593261 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.593101 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.593495 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.593324 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.595351 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.595329 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.595675 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.595656 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8rn4k\"" Jul 18 02:40:10.596696 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.596475 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 02:40:10.598853 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.598834 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.598933 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.598755 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 02:40:10.598933 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.598926 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 02:40:10.599038 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.599011 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.599105 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.599092 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.599247 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.599232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zf9mr\"" Jul 18 02:40:10.599318 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.599300 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 02:40:10.600251 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.600234 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 02:40:10.600529 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.600511 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 02:40:10.600599 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.600565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 02:40:10.600684 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.600514 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 02:40:10.600947 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.600933 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 02:40:10.601086 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.601070 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d9vnz\"" Jul 18 02:40:10.601269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.601202 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 02:40:10.601269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.601236 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 02:40:10.601770 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.601754 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 02:40:10.602237 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.602076 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c57cp\"" Jul 18 02:40:10.602983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.602769 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 02:40:10.604268 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-systemd\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604350 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604280 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7793953-fe81-4ba2-af53-15518dc880e4-host\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.604350 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604304 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.604350 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604317 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-sys\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604350 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-var-lib-kubelet\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604350 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604343 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6db4g\" (UniqueName: \"kubernetes.io/projected/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-kube-api-access-6db4g\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7793953-fe81-4ba2-af53-15518dc880e4-serviceca\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq2wr\" (UniqueName: \"kubernetes.io/projected/a7793953-fe81-4ba2-af53-15518dc880e4-kube-api-access-gq2wr\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysconfig\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-run\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-socket-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604482 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzv5k\" (UniqueName: \"kubernetes.io/projected/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-kube-api-access-nzv5k\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysctl-d\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-lib-modules\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-tmp-dir\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-host\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-tmp\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwrqm\" (UniqueName: \"kubernetes.io/projected/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-kube-api-access-rwrqm\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604654 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-registration-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-sys-fs\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48pbc\" (UniqueName: \"kubernetes.io/projected/4923034a-534b-494c-8350-2e6b0ecdf4bc-kube-api-access-48pbc\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-modprobe-d\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604778 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-kubernetes\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.604897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-hosts-file\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.605460 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604817 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-device-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.605460 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysctl-conf\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.605460 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.604866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-tuned\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.616891 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.616875 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jr8gh" Jul 18 02:40:10.625005 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.624984 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jr8gh" Jul 18 02:40:10.687587 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.687564 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 02:40:10.705812 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705789 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovn-node-metrics-cert\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.705917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5kd5\" (UniqueName: \"kubernetes.io/projected/c9668b8d-42c4-435f-a8a6-619519fd4f93-kube-api-access-r5kd5\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.705917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-systemd\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.705917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7793953-fe81-4ba2-af53-15518dc880e4-host\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.705917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705902 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-systemd\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-cni-netd\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-env-overrides\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.705980 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-systemd\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12234ba8-aa84-40ea-938c-9da3073e3495-host-slash\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7793953-fe81-4ba2-af53-15518dc880e4-host\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba51b136-217a-436d-80ba-8b5332c264fd-kubelet-config\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4279194f-e379-4c62-8447-90668c667b97-konnectivity-ca\") pod \"konnectivity-agent-mqgz7\" (UID: \"4279194f-e379-4c62-8447-90668c667b97\") " pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706086 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-sys\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706125 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-sys\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.706148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706141 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4279194f-e379-4c62-8447-90668c667b97-agent-certs\") pod \"konnectivity-agent-mqgz7\" (UID: \"4279194f-e379-4c62-8447-90668c667b97\") " pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-var-lib-kubelet\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6db4g\" (UniqueName: \"kubernetes.io/projected/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-kube-api-access-6db4g\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7793953-fe81-4ba2-af53-15518dc880e4-serviceca\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706248 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-var-lib-kubelet\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq2wr\" (UniqueName: \"kubernetes.io/projected/a7793953-fe81-4ba2-af53-15518dc880e4-kube-api-access-gq2wr\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-cni-bin\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706345 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj7nl\" (UniqueName: \"kubernetes.io/projected/12234ba8-aa84-40ea-938c-9da3073e3495-kube-api-access-fj7nl\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-os-release\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-cni-binary-copy\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706419 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-k8s-cni-cncf-io\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706446 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-multus-certs\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.706593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706471 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-socket-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzv5k\" (UniqueName: \"kubernetes.io/projected/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-kube-api-access-nzv5k\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysctl-d\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706610 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-socket-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-lib-modules\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-tmp-dir\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovnkube-config\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706669 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysctl-d\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-host\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706718 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-ovn\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706753 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-lib-modules\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-host\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7793953-fe81-4ba2-af53-15518dc880e4-serviceca\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-cnibin\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.707256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706821 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-system-cni-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-netns\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706881 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-registration-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-sys-fs\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-tmp-dir\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-modprobe-d\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706957 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-registration-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.706981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-kubernetes\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-hosts-file\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707041 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-sys-fs\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707047 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-modprobe-d\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.707825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707053 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06678ee4-c1e6-4e29-8c7f-eff88e49a516-cni-binary-copy\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707042 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-kubernetes\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-hosts-file\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-node-log\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-cni-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wgg9\" (UniqueName: \"kubernetes.io/projected/06678ee4-c1e6-4e29-8c7f-eff88e49a516-kube-api-access-7wgg9\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-kubelet\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707242 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707279 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-slash\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707306 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-log-socket\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707322 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-socket-dir-parent\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-conf-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.708453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-run-ovn-kubernetes\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovnkube-script-lib\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707422 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-daemon-config\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707441 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba51b136-217a-436d-80ba-8b5332c264fd-dbus\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysconfig\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysconfig\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-run\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-systemd-units\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-run-netns\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-cnibin\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707632 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-cni-multus\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-run\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/12234ba8-aa84-40ea-938c-9da3073e3495-iptables-alerter-script\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.709078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-kubelet\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-tmp\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707815 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.707849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwrqm\" (UniqueName: \"kubernetes.io/projected/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-kube-api-access-rwrqm\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-var-lib-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvn2s\" (UniqueName: \"kubernetes.io/projected/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-kube-api-access-rvn2s\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708242 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-system-cni-dir\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-os-release\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48pbc\" (UniqueName: \"kubernetes.io/projected/4923034a-534b-494c-8350-2e6b0ecdf4bc-kube-api-access-48pbc\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-cni-bin\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-hostroot\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708399 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-etc-kubernetes\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.709668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-device-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysctl-conf\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-tuned\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-etc-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708604 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4923034a-534b-494c-8350-2e6b0ecdf4bc-device-dir\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.708626 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.708686 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:40:11.208659027 +0000 UTC m=+2.997432783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-sysctl-conf\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.710301 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.708346 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 02:40:10.712364 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.712155 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-etc-tuned\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.712364 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.712169 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-tmp\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.712364 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.712262 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:10.712364 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.712279 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:10.712364 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.712290 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:10.712364 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.712334 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:11.212319779 +0000 UTC m=+3.001093712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:10.714163 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.714116 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzv5k\" (UniqueName: \"kubernetes.io/projected/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-kube-api-access-nzv5k\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:10.714412 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.714391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6db4g\" (UniqueName: \"kubernetes.io/projected/2821e9ad-919b-4e88-a9dd-8f54e8c3b998-kube-api-access-6db4g\") pod \"tuned-wqhjw\" (UID: \"2821e9ad-919b-4e88-a9dd-8f54e8c3b998\") " pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.715693 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.715674 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwrqm\" (UniqueName: \"kubernetes.io/projected/fd825113-9c7b-4f2e-be4d-7d4074bc1c65-kube-api-access-rwrqm\") pod \"node-resolver-7zpc4\" (UID: \"fd825113-9c7b-4f2e-be4d-7d4074bc1c65\") " pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.715925 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.715906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq2wr\" (UniqueName: \"kubernetes.io/projected/a7793953-fe81-4ba2-af53-15518dc880e4-kube-api-access-gq2wr\") pod \"node-ca-9b5gs\" (UID: \"a7793953-fe81-4ba2-af53-15518dc880e4\") " pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.716337 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.716317 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48pbc\" (UniqueName: \"kubernetes.io/projected/4923034a-534b-494c-8350-2e6b0ecdf4bc-kube-api-access-48pbc\") pod \"aws-ebs-csi-driver-node-clz6c\" (UID: \"4923034a-534b-494c-8350-2e6b0ecdf4bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.809757 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809686 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.809757 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/12234ba8-aa84-40ea-938c-9da3073e3495-iptables-alerter-script\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.809757 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-kubelet\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-var-lib-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvn2s\" (UniqueName: \"kubernetes.io/projected/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-kube-api-access-rvn2s\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809808 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-system-cni-dir\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-os-release\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-cni-bin\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809867 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-hostroot\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-etc-kubernetes\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809889 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-var-lib-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809893 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-system-cni-dir\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-etc-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-kubelet\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovn-node-metrics-cert\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-os-release\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.810014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-etc-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.809997 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-hostroot\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810009 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-cni-bin\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5kd5\" (UniqueName: \"kubernetes.io/projected/c9668b8d-42c4-435f-a8a6-619519fd4f93-kube-api-access-r5kd5\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810043 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-etc-kubernetes\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810045 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-systemd\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-cni-netd\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810073 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-systemd\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810091 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-env-overrides\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810112 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12234ba8-aa84-40ea-938c-9da3073e3495-host-slash\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-cni-netd\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba51b136-217a-436d-80ba-8b5332c264fd-kubelet-config\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4279194f-e379-4c62-8447-90668c667b97-konnectivity-ca\") pod \"konnectivity-agent-mqgz7\" (UID: \"4279194f-e379-4c62-8447-90668c667b97\") " pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4279194f-e379-4c62-8447-90668c667b97-agent-certs\") pod \"konnectivity-agent-mqgz7\" (UID: \"4279194f-e379-4c62-8447-90668c667b97\") " pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.810616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba51b136-217a-436d-80ba-8b5332c264fd-kubelet-config\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-cni-bin\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj7nl\" (UniqueName: \"kubernetes.io/projected/12234ba8-aa84-40ea-938c-9da3073e3495-kube-api-access-fj7nl\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-os-release\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/12234ba8-aa84-40ea-938c-9da3073e3495-iptables-alerter-script\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810304 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-cni-binary-copy\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-k8s-cni-cncf-io\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810352 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810359 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-openvswitch\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-multus-certs\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810402 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-os-release\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810431 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-cni-bin\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810444 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-k8s-cni-cncf-io\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-multus-certs\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovnkube-config\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810500 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-ovn\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-cnibin\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-env-overrides\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-system-cni-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-netns\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810603 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12234ba8-aa84-40ea-938c-9da3073e3495-host-slash\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810639 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9668b8d-42c4-435f-a8a6-619519fd4f93-cnibin\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06678ee4-c1e6-4e29-8c7f-eff88e49a516-cni-binary-copy\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4279194f-e379-4c62-8447-90668c667b97-konnectivity-ca\") pod \"konnectivity-agent-mqgz7\" (UID: \"4279194f-e379-4c62-8447-90668c667b97\") " pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.811939 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-node-log\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810693 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-cni-binary-copy\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-cni-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-system-cni-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wgg9\" (UniqueName: \"kubernetes.io/projected/06678ee4-c1e6-4e29-8c7f-eff88e49a516-kube-api-access-7wgg9\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810734 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-cni-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810756 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-run-ovn\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-kubelet\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810786 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.810805 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:10.810852 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:40:11.310838874 +0000 UTC m=+3.099612620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810851 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-slash\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810795 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-slash\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.812530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-log-socket\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.810914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-socket-dir-parent\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811037 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-log-socket\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811058 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9668b8d-42c4-435f-a8a6-619519fd4f93-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovnkube-config\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811124 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-socket-dir-parent\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811158 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-kubelet\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-run-netns\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811256 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-conf-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-conf-dir\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-node-log\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811294 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06678ee4-c1e6-4e29-8c7f-eff88e49a516-cni-binary-copy\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-run-ovn-kubernetes\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovnkube-script-lib\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811344 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-run-ovn-kubernetes\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-daemon-config\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811377 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba51b136-217a-436d-80ba-8b5332c264fd-dbus\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-systemd-units\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-run-netns\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-cnibin\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811466 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba51b136-217a-436d-80ba-8b5332c264fd-dbus\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-cni-multus\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811532 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-host-var-lib-cni-multus\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811575 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-systemd-units\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811612 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-host-run-netns\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811646 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06678ee4-c1e6-4e29-8c7f-eff88e49a516-cnibin\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovnkube-script-lib\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.811883 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06678ee4-c1e6-4e29-8c7f-eff88e49a516-multus-daemon-config\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.812461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-ovn-node-metrics-cert\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.813656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.813461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4279194f-e379-4c62-8447-90668c667b97-agent-certs\") pod \"konnectivity-agent-mqgz7\" (UID: \"4279194f-e379-4c62-8447-90668c667b97\") " pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.817552 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.817530 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj7nl\" (UniqueName: \"kubernetes.io/projected/12234ba8-aa84-40ea-938c-9da3073e3495-kube-api-access-fj7nl\") pod \"iptables-alerter-rq5t9\" (UID: \"12234ba8-aa84-40ea-938c-9da3073e3495\") " pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.817552 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.817547 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5kd5\" (UniqueName: \"kubernetes.io/projected/c9668b8d-42c4-435f-a8a6-619519fd4f93-kube-api-access-r5kd5\") pod \"multus-additional-cni-plugins-x5g6r\" (UID: \"c9668b8d-42c4-435f-a8a6-619519fd4f93\") " pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:10.817822 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.817802 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvn2s\" (UniqueName: \"kubernetes.io/projected/8c0e45f7-67a2-4ef8-8fa6-37c879bfac64-kube-api-access-rvn2s\") pod \"ovnkube-node-4grnx\" (UID: \"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64\") " pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.818617 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.818598 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wgg9\" (UniqueName: \"kubernetes.io/projected/06678ee4-c1e6-4e29-8c7f-eff88e49a516-kube-api-access-7wgg9\") pod \"multus-5sj4d\" (UID: \"06678ee4-c1e6-4e29-8c7f-eff88e49a516\") " pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.889614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.889585 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" Jul 18 02:40:10.898193 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.898176 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" Jul 18 02:40:10.906658 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.906638 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7zpc4" Jul 18 02:40:10.913249 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.913228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9b5gs" Jul 18 02:40:10.920855 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.920841 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rq5t9" Jul 18 02:40:10.928232 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.928217 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:10.935558 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.935538 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:10.942098 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.942080 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5sj4d" Jul 18 02:40:10.947596 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:10.947581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" Jul 18 02:40:11.024177 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.024155 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 02:40:11.214625 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.214552 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:11.214625 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.214621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:11.214809 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.214704 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:11.214809 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.214711 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:11.214809 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.214735 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:11.214809 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.214747 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:11.214809 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.214759 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:40:12.214742017 +0000 UTC m=+4.003515767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:11.214809 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.214783 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:12.214772116 +0000 UTC m=+4.003545866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:11.315234 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.315212 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:11.315370 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.315320 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:11.315408 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.315375 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:40:12.315359888 +0000 UTC m=+4.104133638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:11.438791 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.438761 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9668b8d_42c4_435f_a8a6_619519fd4f93.slice/crio-a3cffb4b300099ab26f2e3c8d3174039f2e953ed8259326b67a18554aa18be6d WatchSource:0}: Error finding container a3cffb4b300099ab26f2e3c8d3174039f2e953ed8259326b67a18554aa18be6d: Status 404 returned error can't find the container with id a3cffb4b300099ab26f2e3c8d3174039f2e953ed8259326b67a18554aa18be6d Jul 18 02:40:11.441564 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.441539 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06678ee4_c1e6_4e29_8c7f_eff88e49a516.slice/crio-a87b76c01f1caefa75d278dc45f2e3f895af3fa49a13f2bccbca5b98b87fb98c WatchSource:0}: Error finding container a87b76c01f1caefa75d278dc45f2e3f895af3fa49a13f2bccbca5b98b87fb98c: Status 404 returned error can't find the container with id a87b76c01f1caefa75d278dc45f2e3f895af3fa49a13f2bccbca5b98b87fb98c Jul 18 02:40:11.444820 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.444800 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12234ba8_aa84_40ea_938c_9da3073e3495.slice/crio-3523124a2869d8bd78293e02be188b37666ddb8aec14e419b90708a48b8d5ba7 WatchSource:0}: Error finding container 3523124a2869d8bd78293e02be188b37666ddb8aec14e419b90708a48b8d5ba7: Status 404 returned error can't find the container with id 3523124a2869d8bd78293e02be188b37666ddb8aec14e419b90708a48b8d5ba7 Jul 18 02:40:11.445426 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.445394 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd825113_9c7b_4f2e_be4d_7d4074bc1c65.slice/crio-0b6b210fd309d7b455ae3db3d62671f7f04b0f181b147cae5af6e7059bc2481d WatchSource:0}: Error finding container 0b6b210fd309d7b455ae3db3d62671f7f04b0f181b147cae5af6e7059bc2481d: Status 404 returned error can't find the container with id 0b6b210fd309d7b455ae3db3d62671f7f04b0f181b147cae5af6e7059bc2481d Jul 18 02:40:11.446231 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.446211 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7793953_fe81_4ba2_af53_15518dc880e4.slice/crio-bb3303248decba6951e89801b1d8399861e99a36b04900bed79bae02e3c0c11c WatchSource:0}: Error finding container bb3303248decba6951e89801b1d8399861e99a36b04900bed79bae02e3c0c11c: Status 404 returned error can't find the container with id bb3303248decba6951e89801b1d8399861e99a36b04900bed79bae02e3c0c11c Jul 18 02:40:11.447508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.447483 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2821e9ad_919b_4e88_a9dd_8f54e8c3b998.slice/crio-f228d3a21d67daede947ecad57a1fd8ecc68694ab40b092f866970f514224fac WatchSource:0}: Error finding container f228d3a21d67daede947ecad57a1fd8ecc68694ab40b092f866970f514224fac: Status 404 returned error can't find the container with id f228d3a21d67daede947ecad57a1fd8ecc68694ab40b092f866970f514224fac Jul 18 02:40:11.448291 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.448271 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0e45f7_67a2_4ef8_8fa6_37c879bfac64.slice/crio-541edf57eb41eefb2679d15db67a94168651a6a25b8ddcdcc9533927612a3aff WatchSource:0}: Error finding container 541edf57eb41eefb2679d15db67a94168651a6a25b8ddcdcc9533927612a3aff: Status 404 returned error can't find the container with id 541edf57eb41eefb2679d15db67a94168651a6a25b8ddcdcc9533927612a3aff Jul 18 02:40:11.449794 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.449778 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4279194f_e379_4c62_8447_90668c667b97.slice/crio-ee01d77f206a18c52dd9ae73fffe111654a1f19f28b62895ac3579863296e8d9 WatchSource:0}: Error finding container ee01d77f206a18c52dd9ae73fffe111654a1f19f28b62895ac3579863296e8d9: Status 404 returned error can't find the container with id ee01d77f206a18c52dd9ae73fffe111654a1f19f28b62895ac3579863296e8d9 Jul 18 02:40:11.450914 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:11.450894 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4923034a_534b_494c_8350_2e6b0ecdf4bc.slice/crio-a0bfd7d1e64aecdab579bdd90d312e06f26a2f67ec3250e50e055c6c22fd01e2 WatchSource:0}: Error finding container a0bfd7d1e64aecdab579bdd90d312e06f26a2f67ec3250e50e055c6c22fd01e2: Status 404 returned error can't find the container with id a0bfd7d1e64aecdab579bdd90d312e06f26a2f67ec3250e50e055c6c22fd01e2 Jul 18 02:40:11.627642 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.627502 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 02:35:10 +0000 UTC" deadline="2028-02-17 05:51:50.393600706 +0000 UTC" Jul 18 02:40:11.627642 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.627641 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13899h11m38.765962019s" Jul 18 02:40:11.751765 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.751713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:11.751859 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.751717 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:11.751909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.751791 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:11.751962 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:11.751902 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:11.756673 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.756655 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5sj4d" event={"ID":"06678ee4-c1e6-4e29-8c7f-eff88e49a516","Type":"ContainerStarted","Data":"a87b76c01f1caefa75d278dc45f2e3f895af3fa49a13f2bccbca5b98b87fb98c"} Jul 18 02:40:11.758364 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.758343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" event={"ID":"1439b4208df4c574b43f6dbf41e494ee","Type":"ContainerStarted","Data":"baf77a23badb1096ede339d9793789aab90a9ccc1769164b97f78ab648d1e2f6"} Jul 18 02:40:11.759319 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.759303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" event={"ID":"2821e9ad-919b-4e88-a9dd-8f54e8c3b998","Type":"ContainerStarted","Data":"f228d3a21d67daede947ecad57a1fd8ecc68694ab40b092f866970f514224fac"} Jul 18 02:40:11.760158 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.760140 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7zpc4" event={"ID":"fd825113-9c7b-4f2e-be4d-7d4074bc1c65","Type":"ContainerStarted","Data":"0b6b210fd309d7b455ae3db3d62671f7f04b0f181b147cae5af6e7059bc2481d"} Jul 18 02:40:11.761160 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.761139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerStarted","Data":"a3cffb4b300099ab26f2e3c8d3174039f2e953ed8259326b67a18554aa18be6d"} Jul 18 02:40:11.762538 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.762519 2560 generic.go:356] "Generic (PLEG): container finished" podID="9ecb42b15a64d1776d35f748981073b4" containerID="f783d776f0073ad761c5c1e216ffae2392bd85e9a50369990452482fa1890fa0" exitCode=0 Jul 18 02:40:11.762624 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.762574 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" event={"ID":"9ecb42b15a64d1776d35f748981073b4","Type":"ContainerDied","Data":"f783d776f0073ad761c5c1e216ffae2392bd85e9a50369990452482fa1890fa0"} Jul 18 02:40:11.763519 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.763502 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" event={"ID":"4923034a-534b-494c-8350-2e6b0ecdf4bc","Type":"ContainerStarted","Data":"a0bfd7d1e64aecdab579bdd90d312e06f26a2f67ec3250e50e055c6c22fd01e2"} Jul 18 02:40:11.765451 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.765430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mqgz7" event={"ID":"4279194f-e379-4c62-8447-90668c667b97","Type":"ContainerStarted","Data":"ee01d77f206a18c52dd9ae73fffe111654a1f19f28b62895ac3579863296e8d9"} Jul 18 02:40:11.766769 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.766754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"541edf57eb41eefb2679d15db67a94168651a6a25b8ddcdcc9533927612a3aff"} Jul 18 02:40:11.767722 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.767697 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9b5gs" event={"ID":"a7793953-fe81-4ba2-af53-15518dc880e4","Type":"ContainerStarted","Data":"bb3303248decba6951e89801b1d8399861e99a36b04900bed79bae02e3c0c11c"} Jul 18 02:40:11.768530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.768504 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rq5t9" event={"ID":"12234ba8-aa84-40ea-938c-9da3073e3495","Type":"ContainerStarted","Data":"3523124a2869d8bd78293e02be188b37666ddb8aec14e419b90708a48b8d5ba7"} Jul 18 02:40:11.772726 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:11.772682 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-139.ec2.internal" podStartSLOduration=2.772665528 podStartE2EDuration="2.772665528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:11.772227451 +0000 UTC m=+3.561001218" watchObservedRunningTime="2026-07-18 02:40:11.772665528 +0000 UTC m=+3.561439294" Jul 18 02:40:12.223895 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.223817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:12.223895 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.223880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:12.224120 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.224041 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:12.224120 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.224061 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:12.224120 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.224072 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:12.224120 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.224122 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:14.224105413 +0000 UTC m=+6.012879159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:12.224480 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.224463 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:12.224556 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.224508 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:40:14.224493747 +0000 UTC m=+6.013267500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:12.324572 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.324540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:12.324719 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.324708 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:12.324772 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.324764 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:40:14.324747364 +0000 UTC m=+6.113521115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:12.628793 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.628755 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 02:35:10 +0000 UTC" deadline="2028-02-16 03:10:17.169820296 +0000 UTC" Jul 18 02:40:12.628793 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.628792 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13872h30m4.541032787s" Jul 18 02:40:12.753872 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.753827 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:12.754047 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:12.753948 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:12.792274 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:12.792244 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" event={"ID":"9ecb42b15a64d1776d35f748981073b4","Type":"ContainerStarted","Data":"af0ff77b01c92876d1b1afe2416718d0a7350176ab7991bd48b156df97f8cf1e"} Jul 18 02:40:13.751604 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:13.751573 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:13.752084 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:13.751711 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:13.752084 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:13.751887 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:13.752084 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:13.751986 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:14.251465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:14.251536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.251656 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.251715 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:40:18.251696767 +0000 UTC m=+10.040470521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.251813 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.251837 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.251849 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:14.252088 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.251903 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:18.251886642 +0000 UTC m=+10.040660410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:14.352512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:14.352473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:14.352677 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.352633 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:14.352753 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.352708 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:40:18.352688563 +0000 UTC m=+10.141462314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:14.751097 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:14.751019 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:14.751245 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:14.751171 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:15.751800 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:15.751767 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:15.752287 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:15.751767 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:15.752287 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:15.751914 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:15.752287 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:15.752015 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:16.752012 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:16.751963 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:16.752432 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:16.752121 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:17.751635 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:17.751602 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:17.751635 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:17.751637 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:17.751887 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:17.751751 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:17.751946 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:17.751914 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:18.285930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:18.286038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.286165 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.286239 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:40:26.286217654 +0000 UTC m=+18.074991403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.286323 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.286344 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.286357 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:18.286526 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.286444 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:26.286385975 +0000 UTC m=+18.075159742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:18.387247 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:18.386996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:18.387247 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.387142 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:18.387247 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.387208 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:40:26.387188106 +0000 UTC m=+18.175961866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:18.752626 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:18.752551 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:18.752784 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:18.752661 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:19.751494 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:19.751412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:19.751917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:19.751532 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:19.751917 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:19.751529 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:19.751917 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:19.751617 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:20.751589 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:20.751555 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:20.752072 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:20.751693 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:21.751310 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:21.751278 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:21.751478 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:21.751278 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:21.751478 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:21.751405 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:21.751575 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:21.751529 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:22.751215 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:22.751182 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:22.751656 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:22.751307 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:23.751960 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:23.751925 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:23.752431 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:23.751925 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:23.752431 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:23.752069 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:23.752431 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:23.752165 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:24.751478 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:24.751451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:24.751651 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:24.751570 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:25.751745 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:25.751711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:25.751745 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:25.751730 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:25.752281 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:25.751813 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:25.752281 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:25.751959 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:26.346272 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:26.346244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:26.346463 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:26.346298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:26.346463 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.346401 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:26.346463 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.346406 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:26.346621 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.346473 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:40:42.346452205 +0000 UTC m=+34.135225959 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:26.346621 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.346414 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:26.346621 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.346513 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:26.346621 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.346569 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:42.34655475 +0000 UTC m=+34.135328498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:26.447262 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:26.447229 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:26.447440 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.447388 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:26.447500 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.447448 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:40:42.44742944 +0000 UTC m=+34.236203188 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:26.752085 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:26.752013 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:26.752502 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:26.752123 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:27.751489 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:27.751450 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:27.751678 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:27.751455 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:27.751678 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:27.751592 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:27.751791 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:27.751668 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:28.752197 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:28.752161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:28.752701 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:28.752273 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:29.750896 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:29.750866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:29.751047 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:29.750866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:29.751047 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:29.750989 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:29.751125 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:29.751051 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:30.751438 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.751182 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:30.751939 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:30.751512 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:30.853025 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.852988 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7zpc4" event={"ID":"fd825113-9c7b-4f2e-be4d-7d4074bc1c65","Type":"ContainerStarted","Data":"1422e20f30c3fd6a5986be0f10de800c1f9fbcc2039baeac15025d75043db402"} Jul 18 02:40:30.854744 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.854721 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9668b8d-42c4-435f-a8a6-619519fd4f93" containerID="0f4d7b110d949758bef79aa7a2d98d801665639c12cf5fc218961439c1ea1f21" exitCode=0 Jul 18 02:40:30.854844 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.854806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerDied","Data":"0f4d7b110d949758bef79aa7a2d98d801665639c12cf5fc218961439c1ea1f21"} Jul 18 02:40:30.856925 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.856897 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" event={"ID":"4923034a-534b-494c-8350-2e6b0ecdf4bc","Type":"ContainerStarted","Data":"2e2e1cdc561fdc4bb1e470725368aa6d935b51a275ac1f6571a1536f5c9027b4"} Jul 18 02:40:30.858775 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.858532 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mqgz7" event={"ID":"4279194f-e379-4c62-8447-90668c667b97","Type":"ContainerStarted","Data":"e4ce5444477b3948a6c69d8cd88fdcc40b2cded0bb6160385663ad76c66b643f"} Jul 18 02:40:30.861476 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.861458 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:40:30.861881 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.861861 2560 generic.go:356] "Generic (PLEG): container finished" podID="8c0e45f7-67a2-4ef8-8fa6-37c879bfac64" containerID="8d99df0b837a8a0c62cc1689d8ff4564db2ae6910fa57c8cf1f6dc6801a10fc6" exitCode=1 Jul 18 02:40:30.861984 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.861927 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"c2941f1901ef536ff38a0cbbfe2ef437932124e7fafc0ec1fd459b63629353b5"} Jul 18 02:40:30.862050 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.862005 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"e81f1f6a29add456d39cd55116682a79c308afe44f0dffd6d67730092004c466"} Jul 18 02:40:30.862050 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.862044 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"4bc68ba1eec6b14cdd71897808fcfb469715e45e90597cc3603bdd8e768f844d"} Jul 18 02:40:30.862152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.862089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerDied","Data":"8d99df0b837a8a0c62cc1689d8ff4564db2ae6910fa57c8cf1f6dc6801a10fc6"} Jul 18 02:40:30.862152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.862136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"cfbed77f6bf287803ab075126475c203f904c6f1e79a22de582a984ae75df3c8"} Jul 18 02:40:30.863571 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.863547 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9b5gs" event={"ID":"a7793953-fe81-4ba2-af53-15518dc880e4","Type":"ContainerStarted","Data":"52085c51f36d08e2ab8ed6344596eba30f9efa081bb9bb55296768845566ad55"} Jul 18 02:40:30.865259 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.865219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5sj4d" event={"ID":"06678ee4-c1e6-4e29-8c7f-eff88e49a516","Type":"ContainerStarted","Data":"ab1875c9849323fb5b9a6b4af1c74c3c533cc55b126eda75994210ec3566cad9"} Jul 18 02:40:30.866945 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.866903 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7zpc4" podStartSLOduration=4.291638718 podStartE2EDuration="22.866888268s" totalImagesPullingTime="18.57524955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:30.866707725 +0000 UTC m=+22.655481495" watchObservedRunningTime="2026-07-18 02:40:30.866888268 +0000 UTC m=+22.655662037" Jul 18 02:40:30.867071 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.866958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" event={"ID":"2821e9ad-919b-4e88-a9dd-8f54e8c3b998","Type":"ContainerStarted","Data":"9dc63a8cbb4f135916d8062b1642a483abf93ce7a90326630c9321f5c42e9b23"} Jul 18 02:40:30.867118 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.867060 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-139.ec2.internal" podStartSLOduration=21.867053824 podStartE2EDuration="21.867053824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:12.806853623 +0000 UTC m=+4.595627392" watchObservedRunningTime="2026-07-18 02:40:30.867053824 +0000 UTC m=+22.655827595" Jul 18 02:40:30.880513 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.880460 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mqgz7" podStartSLOduration=12.735213978 podStartE2EDuration="22.880444804s" totalImagesPullingTime="10.145230826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:30.879863037 +0000 UTC m=+22.668636809" watchObservedRunningTime="2026-07-18 02:40:30.880444804 +0000 UTC m=+22.669218575" Jul 18 02:40:30.896383 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.896330 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9b5gs" podStartSLOduration=4.348695759 podStartE2EDuration="22.896311676s" totalImagesPullingTime="18.547615917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:30.89593678 +0000 UTC m=+22.684710551" watchObservedRunningTime="2026-07-18 02:40:30.896311676 +0000 UTC m=+22.685085447" Jul 18 02:40:30.912874 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:30.912826 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5sj4d" podStartSLOduration=4.327209721 podStartE2EDuration="22.912810166s" totalImagesPullingTime="18.585600445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:30.912549209 +0000 UTC m=+22.701322980" watchObservedRunningTime="2026-07-18 02:40:30.912810166 +0000 UTC m=+22.701583939" Jul 18 02:40:31.686907 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.686884 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 02:40:31.751433 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.751373 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:31.751554 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:31.751482 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:31.751884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.751572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:31.751884 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:31.751668 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:31.873083 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.873055 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:40:31.873561 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.873537 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"f72d78d8c28c3b33c015d54f16e3fe6cdca19b8c34a7ce8d9ac346f29bb0c40b"} Jul 18 02:40:31.875145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.875114 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rq5t9" event={"ID":"12234ba8-aa84-40ea-938c-9da3073e3495","Type":"ContainerStarted","Data":"6c62eb1c2b3e67285b27a2e18a824f42ec7b1485c7d597576acf9cbb341cbb51"} Jul 18 02:40:31.876887 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.876857 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" event={"ID":"4923034a-534b-494c-8350-2e6b0ecdf4bc","Type":"ContainerStarted","Data":"75007658c51d33fd6e46b59a66836698f109d71afe85d7a2f79fbed3389bacb5"} Jul 18 02:40:31.888983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.888926 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wqhjw" podStartSLOduration=5.314497192 podStartE2EDuration="23.888907444s" totalImagesPullingTime="18.574410252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:30.952853549 +0000 UTC m=+22.741627332" watchObservedRunningTime="2026-07-18 02:40:31.888907444 +0000 UTC m=+23.677681214" Jul 18 02:40:31.889169 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:31.889148 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rq5t9" podStartSLOduration=5.313162361 podStartE2EDuration="23.889140598s" totalImagesPullingTime="18.575978237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:31.888795614 +0000 UTC m=+23.677569384" watchObservedRunningTime="2026-07-18 02:40:31.889140598 +0000 UTC m=+23.677914368" Jul 18 02:40:32.667783 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:32.667675 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T02:40:31.686903938Z","UUID":"a8b4309b-7b29-472c-95e2-3c6147d895cc","Handler":null,"Name":"","Endpoint":""} Jul 18 02:40:32.669354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:32.669331 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 02:40:32.669480 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:32.669365 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 02:40:32.751896 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:32.751868 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:32.752328 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:32.752015 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:32.882115 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:32.882073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" event={"ID":"4923034a-534b-494c-8350-2e6b0ecdf4bc","Type":"ContainerStarted","Data":"2a9e5bbb52e6ff9610ba0c207e53e2c06162ebaad5b2b3aa1e38478a046b1221"} Jul 18 02:40:32.899624 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:32.899579 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-clz6c" podStartSLOduration=4.028059384 podStartE2EDuration="24.899564657s" totalImagesPullingTime="20.871505273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:32.898988744 +0000 UTC m=+24.687762515" watchObservedRunningTime="2026-07-18 02:40:32.899564657 +0000 UTC m=+24.688338470" Jul 18 02:40:33.751788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:33.751757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:33.751998 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:33.751759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:33.751998 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:33.751870 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:33.752473 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:33.751997 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:33.887375 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:33.887344 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:40:33.887809 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:33.887785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"58b3c24d6900feb5366eac65c1aca007609365c7c1d1f0af6105ff30358b04d8"} Jul 18 02:40:34.752006 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:34.751746 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:34.752495 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:34.752117 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:35.751234 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.751172 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:35.751348 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.751172 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:35.751348 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:35.751273 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:35.751348 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:35.751336 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:35.765475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.765450 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:35.766268 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.766213 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:35.893610 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.893581 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9668b8d-42c4-435f-a8a6-619519fd4f93" containerID="f33df18fbfea57d58c19ec5aff2a626f1e19ef536984b13409b70f43718f8d2d" exitCode=0 Jul 18 02:40:35.893723 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.893664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerDied","Data":"f33df18fbfea57d58c19ec5aff2a626f1e19ef536984b13409b70f43718f8d2d"} Jul 18 02:40:35.900240 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.900224 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:40:35.900571 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.900548 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"bcb166c4a8c98a1f5f90298ce7629051c0daa20c900382ea80dfe705c8194add"} Jul 18 02:40:35.900798 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.900765 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:35.900866 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.900841 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:35.900917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.900889 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:35.900984 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.900924 2560 scope.go:117] "RemoveContainer" containerID="8d99df0b837a8a0c62cc1689d8ff4564db2ae6910fa57c8cf1f6dc6801a10fc6" Jul 18 02:40:35.915785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.915766 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:35.915954 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:35.915938 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:40:36.751948 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:36.751770 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:36.752149 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:36.752059 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:36.905724 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:36.905696 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:40:36.906062 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:36.906007 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" event={"ID":"8c0e45f7-67a2-4ef8-8fa6-37c879bfac64","Type":"ContainerStarted","Data":"c1aa3e9132dbd08e04cfdafde391db7531b2f382078fb8c74640d62946d9bbb7"} Jul 18 02:40:36.934874 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:36.934675 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" podStartSLOduration=10.078262811 podStartE2EDuration="28.934655709s" totalImagesPullingTime="18.856392898s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:36.934233687 +0000 UTC m=+28.723007483" watchObservedRunningTime="2026-07-18 02:40:36.934655709 +0000 UTC m=+28.723429479" Jul 18 02:40:37.120678 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.120621 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jzqvv"] Jul 18 02:40:37.120786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.120742 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:37.120836 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:37.120819 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:37.123425 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.123398 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nswl4"] Jul 18 02:40:37.123544 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.123501 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:37.123635 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:37.123612 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:37.123942 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.123920 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ltr2b"] Jul 18 02:40:37.124062 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.124018 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:37.124123 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:37.124089 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:37.910691 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.910661 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9668b8d-42c4-435f-a8a6-619519fd4f93" containerID="31e3e9689b3e40807034da7e1be44601a0e58af3b40bfaf51f6b0cbb7885925b" exitCode=0 Jul 18 02:40:37.911126 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:37.910747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerDied","Data":"31e3e9689b3e40807034da7e1be44601a0e58af3b40bfaf51f6b0cbb7885925b"} Jul 18 02:40:38.752717 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:38.752686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:38.752876 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:38.752782 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:38.752876 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:38.752813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:38.752876 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:38.752816 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:38.753010 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:38.752891 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:38.753010 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:38.752987 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:39.227261 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:39.227019 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:39.227669 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:39.227411 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 18 02:40:39.227669 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:39.227597 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mqgz7" Jul 18 02:40:39.917035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:39.916997 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9668b8d-42c4-435f-a8a6-619519fd4f93" containerID="9001a371468e94dde72e4180adc0aceddf7abb70ff08d5d0fb69b2b6ffbdaebc" exitCode=0 Jul 18 02:40:39.917172 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:39.917075 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerDied","Data":"9001a371468e94dde72e4180adc0aceddf7abb70ff08d5d0fb69b2b6ffbdaebc"} Jul 18 02:40:40.751000 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:40.750956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:40.751495 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:40.750956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:40.751495 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:40.751084 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:40.751495 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:40.751105 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:40.751495 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:40.751223 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:40.751495 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:40.751310 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:42.068854 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.068831 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7zpc4_fd825113-9c7b-4f2e-be4d-7d4074bc1c65/dns-node-resolver/0.log" Jul 18 02:40:42.367381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.367297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:42.367381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.367367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:42.367576 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.367469 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 02:40:42.367576 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.367494 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 02:40:42.367576 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.367508 2560 projected.go:196] Error preparing data for projected volume kube-api-access-6pbhz for pod openshift-network-diagnostics/network-check-target-ltr2b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:42.367576 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.367471 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:42.367748 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.367564 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz podName:e22fbac2-5236-4ef7-801c-8ee05261b455 nodeName:}" failed. No retries permitted until 2026-07-18 02:41:14.367544335 +0000 UTC m=+66.156318096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6pbhz" (UniqueName: "kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz") pod "network-check-target-ltr2b" (UID: "e22fbac2-5236-4ef7-801c-8ee05261b455") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 02:40:42.367748 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.367642 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs podName:e42b787c-d4c0-4bf0-bf67-bc10adb618ff nodeName:}" failed. No retries permitted until 2026-07-18 02:41:14.367622422 +0000 UTC m=+66.156396175 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs") pod "network-metrics-daemon-jzqvv" (UID: "e42b787c-d4c0-4bf0-bf67-bc10adb618ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 02:40:42.468152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.468128 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:42.468319 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.468245 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:42.468319 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.468288 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret podName:ba51b136-217a-436d-80ba-8b5332c264fd nodeName:}" failed. No retries permitted until 2026-07-18 02:41:14.46827613 +0000 UTC m=+66.257049878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret") pod "global-pull-secret-syncer-nswl4" (UID: "ba51b136-217a-436d-80ba-8b5332c264fd") : object "kube-system"/"original-pull-secret" not registered Jul 18 02:40:42.751470 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.751433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:42.751652 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.751434 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:42.751652 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.751568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:42.751652 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.751568 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltr2b" podUID="e22fbac2-5236-4ef7-801c-8ee05261b455" Jul 18 02:40:42.751816 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.751668 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jzqvv" podUID="e42b787c-d4c0-4bf0-bf67-bc10adb618ff" Jul 18 02:40:42.751816 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:42.751746 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nswl4" podUID="ba51b136-217a-436d-80ba-8b5332c264fd" Jul 18 02:40:42.970040 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.969955 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-139.ec2.internal" event="NodeReady" Jul 18 02:40:42.970181 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:42.970104 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 02:40:43.012260 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.012228 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wbsjv"] Jul 18 02:40:43.015079 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.015054 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.015259 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.015151 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9dw4n"] Jul 18 02:40:43.016942 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.016918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.017683 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.017652 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 02:40:43.017683 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.017656 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nv5k\"" Jul 18 02:40:43.017864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.017847 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 02:40:43.017909 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.017879 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 02:40:43.018132 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.018118 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 02:40:43.019754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.019733 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 02:40:43.020076 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.020055 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pjzzd\"" Jul 18 02:40:43.020208 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.020058 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 02:40:43.020318 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.020294 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 02:40:43.029619 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.029363 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wbsjv"] Jul 18 02:40:43.030155 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.030137 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9dw4n"] Jul 18 02:40:43.033366 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.033346 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5nt6c"] Jul 18 02:40:43.035903 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.035886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.038443 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.038426 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 02:40:43.038549 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.038532 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4pdzm\"" Jul 18 02:40:43.038686 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.038668 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 02:40:43.044832 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.044812 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5nt6c"] Jul 18 02:40:43.173587 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/02c7cf7d-935f-4bcc-8905-bd040c739002-crio-socket\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.173587 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/02c7cf7d-935f-4bcc-8905-bd040c739002-data-volume\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvxtf\" (UniqueName: \"kubernetes.io/projected/a38f5409-3fa9-4a9e-9ada-4371a96c516e-kube-api-access-kvxtf\") pod \"ingress-canary-9dw4n\" (UID: \"a38f5409-3fa9-4a9e-9ada-4371a96c516e\") " pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e317b0e3-93bf-4318-a2ad-8d683762e301-config-volume\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqpwq\" (UniqueName: \"kubernetes.io/projected/e317b0e3-93bf-4318-a2ad-8d683762e301-kube-api-access-kqpwq\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a38f5409-3fa9-4a9e-9ada-4371a96c516e-cert\") pod \"ingress-canary-9dw4n\" (UID: \"a38f5409-3fa9-4a9e-9ada-4371a96c516e\") " pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngs8b\" (UniqueName: \"kubernetes.io/projected/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-api-access-ngs8b\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.173998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e317b0e3-93bf-4318-a2ad-8d683762e301-tmp-dir\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.174367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.174032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.274420 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e317b0e3-93bf-4318-a2ad-8d683762e301-tmp-dir\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.274593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.274593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/02c7cf7d-935f-4bcc-8905-bd040c739002-crio-socket\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.274593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274495 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/02c7cf7d-935f-4bcc-8905-bd040c739002-data-volume\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.274593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274517 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.274593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.274593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvxtf\" (UniqueName: \"kubernetes.io/projected/a38f5409-3fa9-4a9e-9ada-4371a96c516e-kube-api-access-kvxtf\") pod \"ingress-canary-9dw4n\" (UID: \"a38f5409-3fa9-4a9e-9ada-4371a96c516e\") " pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e317b0e3-93bf-4318-a2ad-8d683762e301-config-volume\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.274620 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqpwq\" (UniqueName: \"kubernetes.io/projected/e317b0e3-93bf-4318-a2ad-8d683762e301-kube-api-access-kqpwq\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a38f5409-3fa9-4a9e-9ada-4371a96c516e-cert\") pod \"ingress-canary-9dw4n\" (UID: \"a38f5409-3fa9-4a9e-9ada-4371a96c516e\") " pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.274693 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.274699 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:43.774679936 +0000 UTC m=+35.563453701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : configmap "kube-rbac-proxy" not found Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/02c7cf7d-935f-4bcc-8905-bd040c739002-crio-socket\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e317b0e3-93bf-4318-a2ad-8d683762e301-tmp-dir\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.274779 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls podName:e317b0e3-93bf-4318-a2ad-8d683762e301 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:43.774761025 +0000 UTC m=+35.563534776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls") pod "dns-default-5nt6c" (UID: "e317b0e3-93bf-4318-a2ad-8d683762e301") : secret "dns-default-metrics-tls" not found Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngs8b\" (UniqueName: \"kubernetes.io/projected/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-api-access-ngs8b\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.274885 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.274826 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/02c7cf7d-935f-4bcc-8905-bd040c739002-data-volume\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.275351 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.274901 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 02:40:43.275351 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.274951 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:43.774937533 +0000 UTC m=+35.563711285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : secret "insights-runtime-extractor-tls" not found Jul 18 02:40:43.275351 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.275257 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e317b0e3-93bf-4318-a2ad-8d683762e301-config-volume\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.279444 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.279421 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a38f5409-3fa9-4a9e-9ada-4371a96c516e-cert\") pod \"ingress-canary-9dw4n\" (UID: \"a38f5409-3fa9-4a9e-9ada-4371a96c516e\") " pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.285588 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.285567 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvxtf\" (UniqueName: \"kubernetes.io/projected/a38f5409-3fa9-4a9e-9ada-4371a96c516e-kube-api-access-kvxtf\") pod \"ingress-canary-9dw4n\" (UID: \"a38f5409-3fa9-4a9e-9ada-4371a96c516e\") " pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.285781 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.285761 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngs8b\" (UniqueName: \"kubernetes.io/projected/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-api-access-ngs8b\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.285882 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.285809 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqpwq\" (UniqueName: \"kubernetes.io/projected/e317b0e3-93bf-4318-a2ad-8d683762e301-kube-api-access-kqpwq\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.335458 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.335433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9dw4n" Jul 18 02:40:43.337833 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.337807 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-z5xbp"] Jul 18 02:40:43.340518 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.340383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.343662 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.343643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 18 02:40:43.343916 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.343896 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 18 02:40:43.344293 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.344202 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 18 02:40:43.344293 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.344220 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 18 02:40:43.344293 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.344254 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wc8x7\"" Jul 18 02:40:43.393435 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.393409 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-z5xbp"] Jul 18 02:40:43.449441 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.449413 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9b5gs_a7793953-fe81-4ba2-af53-15518dc880e4/node-ca/0.log" Jul 18 02:40:43.476148 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.476121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g9q2\" (UniqueName: \"kubernetes.io/projected/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-kube-api-access-9g9q2\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.476281 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.476223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-signing-key\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.476341 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.476312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-signing-cabundle\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.576999 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.576925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-signing-key\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.577110 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.577010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-signing-cabundle\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.577110 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.577038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g9q2\" (UniqueName: \"kubernetes.io/projected/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-kube-api-access-9g9q2\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.577700 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.577681 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-signing-cabundle\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.579584 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.579564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-signing-key\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.585961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.585941 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g9q2\" (UniqueName: \"kubernetes.io/projected/c6fc83c6-7105-4dd9-87a3-e126178e8b1d-kube-api-access-9g9q2\") pod \"service-ca-58598695f7-z5xbp\" (UID: \"c6fc83c6-7105-4dd9-87a3-e126178e8b1d\") " pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.652147 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.652114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-z5xbp" Jul 18 02:40:43.777903 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.777862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.777903 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.777907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:43.778143 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.778009 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 02:40:43.778143 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.778033 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 02:40:43.778143 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.778089 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:44.7780707 +0000 UTC m=+36.566844450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : secret "insights-runtime-extractor-tls" not found Jul 18 02:40:43.778143 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.778108 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:44.77809774 +0000 UTC m=+36.566871491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : configmap "kube-rbac-proxy" not found Jul 18 02:40:43.778143 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:43.778135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:43.778376 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.778237 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 02:40:43.778376 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:43.778273 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls podName:e317b0e3-93bf-4318-a2ad-8d683762e301 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:44.778263038 +0000 UTC m=+36.567036791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls") pod "dns-default-5nt6c" (UID: "e317b0e3-93bf-4318-a2ad-8d683762e301") : secret "dns-default-metrics-tls" not found Jul 18 02:40:44.164005 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.163845 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-z5xbp"] Jul 18 02:40:44.166243 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:44.166217 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fc83c6_7105_4dd9_87a3_e126178e8b1d.slice/crio-7b6aca2d813faab2be3a3f800b4ebfb92b97ce13a2b74559410b0a7ff272762e WatchSource:0}: Error finding container 7b6aca2d813faab2be3a3f800b4ebfb92b97ce13a2b74559410b0a7ff272762e: Status 404 returned error can't find the container with id 7b6aca2d813faab2be3a3f800b4ebfb92b97ce13a2b74559410b0a7ff272762e Jul 18 02:40:44.171209 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.171190 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9dw4n"] Jul 18 02:40:44.174520 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:44.174496 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38f5409_3fa9_4a9e_9ada_4371a96c516e.slice/crio-f27d6832d88c24445bfa51b0ad7e0c03f148b81b18fc9888699a50a86e7e19aa WatchSource:0}: Error finding container f27d6832d88c24445bfa51b0ad7e0c03f148b81b18fc9888699a50a86e7e19aa: Status 404 returned error can't find the container with id f27d6832d88c24445bfa51b0ad7e0c03f148b81b18fc9888699a50a86e7e19aa Jul 18 02:40:44.751759 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.751405 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:40:44.751759 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.751405 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:40:44.751759 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.751550 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:40:44.754280 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.754259 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 02:40:44.754389 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.754335 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 02:40:44.754493 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.754480 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 02:40:44.755345 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.755324 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 02:40:44.755562 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.755541 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzcn2\"" Jul 18 02:40:44.755754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.755717 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zt6zk\"" Jul 18 02:40:44.788041 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.788019 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:44.788184 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.788070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:44.788184 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.788098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:44.788184 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:44.788165 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 02:40:44.788184 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:44.788173 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 02:40:44.788383 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:44.788194 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 02:40:44.788383 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:44.788228 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:46.788206899 +0000 UTC m=+38.576980658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : configmap "kube-rbac-proxy" not found Jul 18 02:40:44.788383 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:44.788284 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls podName:e317b0e3-93bf-4318-a2ad-8d683762e301 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:46.788273994 +0000 UTC m=+38.577047745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls") pod "dns-default-5nt6c" (UID: "e317b0e3-93bf-4318-a2ad-8d683762e301") : secret "dns-default-metrics-tls" not found Jul 18 02:40:44.788383 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:44.788300 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:46.788292421 +0000 UTC m=+38.577066168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : secret "insights-runtime-extractor-tls" not found Jul 18 02:40:44.933300 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.933265 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9668b8d-42c4-435f-a8a6-619519fd4f93" containerID="06b1a3218c1419ffb41cd62dc05f4d3da3cf0b186032aa9dbd48e7f32828241f" exitCode=0 Jul 18 02:40:44.933469 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.933348 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerDied","Data":"06b1a3218c1419ffb41cd62dc05f4d3da3cf0b186032aa9dbd48e7f32828241f"} Jul 18 02:40:44.934931 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.934906 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9dw4n" event={"ID":"a38f5409-3fa9-4a9e-9ada-4371a96c516e","Type":"ContainerStarted","Data":"f27d6832d88c24445bfa51b0ad7e0c03f148b81b18fc9888699a50a86e7e19aa"} Jul 18 02:40:44.936172 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:44.936148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-z5xbp" event={"ID":"c6fc83c6-7105-4dd9-87a3-e126178e8b1d","Type":"ContainerStarted","Data":"7b6aca2d813faab2be3a3f800b4ebfb92b97ce13a2b74559410b0a7ff272762e"} Jul 18 02:40:45.943467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:45.943431 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9668b8d-42c4-435f-a8a6-619519fd4f93" containerID="72c5160262bd0b9b6b850f47fbb5d1ce0d135cdc9fe727258213120e2d0078d7" exitCode=0 Jul 18 02:40:45.943997 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:45.943481 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerDied","Data":"72c5160262bd0b9b6b850f47fbb5d1ce0d135cdc9fe727258213120e2d0078d7"} Jul 18 02:40:46.804497 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:46.804472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:46.804629 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:46.804512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:46.804629 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:46.804530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:46.804629 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:46.804605 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 02:40:46.804629 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:46.804620 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 02:40:46.804629 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:46.804629 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 02:40:46.804790 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:46.804673 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls podName:e317b0e3-93bf-4318-a2ad-8d683762e301 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:50.804656924 +0000 UTC m=+42.593430686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls") pod "dns-default-5nt6c" (UID: "e317b0e3-93bf-4318-a2ad-8d683762e301") : secret "dns-default-metrics-tls" not found Jul 18 02:40:46.804790 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:46.804688 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:50.804680436 +0000 UTC m=+42.593454183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : secret "insights-runtime-extractor-tls" not found Jul 18 02:40:46.804790 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:46.804698 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:50.804693296 +0000 UTC m=+42.593467044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : configmap "kube-rbac-proxy" not found Jul 18 02:40:47.951548 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:47.951331 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" event={"ID":"c9668b8d-42c4-435f-a8a6-619519fd4f93","Type":"ContainerStarted","Data":"7f2e4af6aa21bb49ca2ee3c9346462e0c8c33b954c91b7bf48633f26c6bfce24"} Jul 18 02:40:47.952854 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:47.952832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9dw4n" event={"ID":"a38f5409-3fa9-4a9e-9ada-4371a96c516e","Type":"ContainerStarted","Data":"bce8dcfea856d0cf84bf3756818fcbee9472ec4278514a942882c2698abb2d18"} Jul 18 02:40:47.954195 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:47.954173 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-z5xbp" event={"ID":"c6fc83c6-7105-4dd9-87a3-e126178e8b1d","Type":"ContainerStarted","Data":"8a533b437ea93a0eb4149a096c6b96b611055d50143e3939890c7a049ea4e05a"} Jul 18 02:40:47.976030 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:47.975991 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x5g6r" podStartSLOduration=10.868961353 podStartE2EDuration="39.975956786s" totalImagesPullingTime="29.106995433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:47.975610166 +0000 UTC m=+39.764383938" watchObservedRunningTime="2026-07-18 02:40:47.975956786 +0000 UTC m=+39.764730550" Jul 18 02:40:47.992942 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:47.992900 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-z5xbp" podStartSLOduration=2.161769503 podStartE2EDuration="4.992888368s" totalImagesPullingTime="2.831118865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:47.992074586 +0000 UTC m=+39.780848369" watchObservedRunningTime="2026-07-18 02:40:47.992888368 +0000 UTC m=+39.781662129" Jul 18 02:40:48.007798 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:48.007762 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9dw4n" podStartSLOduration=3.17894787 podStartE2EDuration="6.007751191s" totalImagesPullingTime="2.828803321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:40:48.007635552 +0000 UTC m=+39.796409322" watchObservedRunningTime="2026-07-18 02:40:48.007751191 +0000 UTC m=+39.796524960" Jul 18 02:40:50.833365 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:50.833329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:50.833369 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:50.833394 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:50.833464 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:50.833526 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:58.833507083 +0000 UTC m=+50.622280848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : configmap "kube-rbac-proxy" not found Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:50.833528 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:50.833468 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:50.833561 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:58.833552171 +0000 UTC m=+50.622325918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : secret "insights-runtime-extractor-tls" not found Jul 18 02:40:50.833909 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:50.833589 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls podName:e317b0e3-93bf-4318-a2ad-8d683762e301 nodeName:}" failed. No retries permitted until 2026-07-18 02:40:58.833574283 +0000 UTC m=+50.622348030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls") pod "dns-default-5nt6c" (UID: "e317b0e3-93bf-4318-a2ad-8d683762e301") : secret "dns-default-metrics-tls" not found Jul 18 02:40:58.891718 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:58.891681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:58.891718 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:58.891717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:58.892209 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:58.891775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:58.892209 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:58.891892 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 02:40:58.892209 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:40:58.891989 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls podName:02c7cf7d-935f-4bcc-8905-bd040c739002 nodeName:}" failed. No retries permitted until 2026-07-18 02:41:14.891948695 +0000 UTC m=+66.680722461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wbsjv" (UID: "02c7cf7d-935f-4bcc-8905-bd040c739002") : secret "insights-runtime-extractor-tls" not found Jul 18 02:40:58.892314 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:58.892292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02c7cf7d-935f-4bcc-8905-bd040c739002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:40:58.895698 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:58.895672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e317b0e3-93bf-4318-a2ad-8d683762e301-metrics-tls\") pod \"dns-default-5nt6c\" (UID: \"e317b0e3-93bf-4318-a2ad-8d683762e301\") " pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:58.944713 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:58.944693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5nt6c" Jul 18 02:40:59.076355 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:59.076332 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5nt6c"] Jul 18 02:40:59.079254 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:40:59.079226 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode317b0e3_93bf_4318_a2ad_8d683762e301.slice/crio-cef832fcc5d02eb7c655918e5b4968bcd9d39864fa91acf3ad3d512e833401ce WatchSource:0}: Error finding container cef832fcc5d02eb7c655918e5b4968bcd9d39864fa91acf3ad3d512e833401ce: Status 404 returned error can't find the container with id cef832fcc5d02eb7c655918e5b4968bcd9d39864fa91acf3ad3d512e833401ce Jul 18 02:40:59.983421 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:40:59.983375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5nt6c" event={"ID":"e317b0e3-93bf-4318-a2ad-8d683762e301","Type":"ContainerStarted","Data":"cef832fcc5d02eb7c655918e5b4968bcd9d39864fa91acf3ad3d512e833401ce"} Jul 18 02:41:01.988593 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:01.988559 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5nt6c" event={"ID":"e317b0e3-93bf-4318-a2ad-8d683762e301","Type":"ContainerStarted","Data":"76451afc9d06da265b5c513d788f563f8e11b2ff516ec61263ac70b4e4646f87"} Jul 18 02:41:01.989094 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:01.988609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5nt6c" event={"ID":"e317b0e3-93bf-4318-a2ad-8d683762e301","Type":"ContainerStarted","Data":"245a37f27ff9a3d9bf6ba8548da48d4f6213adea4e4cc2db4a842b3d8f2af12c"} Jul 18 02:41:01.989094 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:01.988698 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5nt6c" Jul 18 02:41:02.004472 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:02.004429 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5nt6c" podStartSLOduration=17.107732354 podStartE2EDuration="19.004417148s" totalImagesPullingTime="1.896684794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:02.0043108 +0000 UTC m=+53.793084561" watchObservedRunningTime="2026-07-18 02:41:02.004417148 +0000 UTC m=+53.793190918" Jul 18 02:41:07.241918 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.241885 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx"] Jul 18 02:41:07.244239 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.244222 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.248672 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.248629 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7gcbm\"" Jul 18 02:41:07.248672 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.248663 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 02:41:07.248879 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.248866 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 02:41:07.249652 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.249634 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 02:41:07.253692 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.253671 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 02:41:07.257594 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.257576 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx"] Jul 18 02:41:07.346915 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.346884 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8448ca3a-17a8-43dc-afc7-4931b4f83318-image-registry-private-configuration\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.346915 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.346916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8448ca3a-17a8-43dc-afc7-4931b4f83318-installation-pull-secrets\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.347108 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.346950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8448ca3a-17a8-43dc-afc7-4931b4f83318-registry-certificates\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.347108 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.347031 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-registry-tls\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.347108 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.347095 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8448ca3a-17a8-43dc-afc7-4931b4f83318-trusted-ca\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.347231 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.347111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-bound-sa-token\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.347231 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.347159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d6c2\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-kube-api-access-5d6c2\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.347231 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.347183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8448ca3a-17a8-43dc-afc7-4931b4f83318-ca-trust-extracted\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448173 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8448ca3a-17a8-43dc-afc7-4931b4f83318-registry-certificates\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448325 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-registry-tls\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448325 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8448ca3a-17a8-43dc-afc7-4931b4f83318-trusted-ca\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448325 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-bound-sa-token\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448325 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6c2\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-kube-api-access-5d6c2\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8448ca3a-17a8-43dc-afc7-4931b4f83318-ca-trust-extracted\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8448ca3a-17a8-43dc-afc7-4931b4f83318-image-registry-private-configuration\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8448ca3a-17a8-43dc-afc7-4931b4f83318-installation-pull-secrets\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.448920 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.448890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8448ca3a-17a8-43dc-afc7-4931b4f83318-ca-trust-extracted\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.449231 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.449204 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8448ca3a-17a8-43dc-afc7-4931b4f83318-registry-certificates\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.449402 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.449381 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8448ca3a-17a8-43dc-afc7-4931b4f83318-trusted-ca\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.451085 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.451060 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8448ca3a-17a8-43dc-afc7-4931b4f83318-image-registry-private-configuration\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.451085 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.451078 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-registry-tls\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.451222 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.451124 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8448ca3a-17a8-43dc-afc7-4931b4f83318-installation-pull-secrets\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.458465 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.458442 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-bound-sa-token\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.458707 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.458691 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6c2\" (UniqueName: \"kubernetes.io/projected/8448ca3a-17a8-43dc-afc7-4931b4f83318-kube-api-access-5d6c2\") pod \"image-registry-5b9fb5fd8d-pmrlx\" (UID: \"8448ca3a-17a8-43dc-afc7-4931b4f83318\") " pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.555108 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.555004 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:07.702195 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.702163 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx"] Jul 18 02:41:07.705651 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:07.705627 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8448ca3a_17a8_43dc_afc7_4931b4f83318.slice/crio-4e6c30b64364d9c3d151a2116bdfdad86296a655a3fb4f160297c043ae72a806 WatchSource:0}: Error finding container 4e6c30b64364d9c3d151a2116bdfdad86296a655a3fb4f160297c043ae72a806: Status 404 returned error can't find the container with id 4e6c30b64364d9c3d151a2116bdfdad86296a655a3fb4f160297c043ae72a806 Jul 18 02:41:07.925467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:07.925404 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4grnx" Jul 18 02:41:08.005503 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:08.005469 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" event={"ID":"8448ca3a-17a8-43dc-afc7-4931b4f83318","Type":"ContainerStarted","Data":"f5cfee372b3502e61376adb951a0b318a9142632a22b5f1ea91694282c775c8a"} Jul 18 02:41:08.005686 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:08.005516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" event={"ID":"8448ca3a-17a8-43dc-afc7-4931b4f83318","Type":"ContainerStarted","Data":"4e6c30b64364d9c3d151a2116bdfdad86296a655a3fb4f160297c043ae72a806"} Jul 18 02:41:08.005686 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:08.005606 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:08.025489 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:08.025433 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" podStartSLOduration=1.025415397 podStartE2EDuration="1.025415397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:08.024561258 +0000 UTC m=+59.813335039" watchObservedRunningTime="2026-07-18 02:41:08.025415397 +0000 UTC m=+59.814189167" Jul 18 02:41:11.992378 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:11.992348 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5nt6c" Jul 18 02:41:12.591853 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.591821 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9"] Jul 18 02:41:12.595173 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.595156 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.597490 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.597468 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 18 02:41:12.598454 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.598433 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 02:41:12.598454 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.598454 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 02:41:12.598601 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.598466 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hqcm7\"" Jul 18 02:41:12.598601 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.598455 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 02:41:12.598601 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.598471 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 18 02:41:12.605797 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.605772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9"] Jul 18 02:41:12.689583 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.689562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf60079b-1b6e-42a5-bc0f-af53a11090dd-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.689671 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.689593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf60079b-1b6e-42a5-bc0f-af53a11090dd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.689671 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.689621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf60079b-1b6e-42a5-bc0f-af53a11090dd-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.689740 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.689701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gr8w\" (UniqueName: \"kubernetes.io/projected/cf60079b-1b6e-42a5-bc0f-af53a11090dd-kube-api-access-8gr8w\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.790876 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.790854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf60079b-1b6e-42a5-bc0f-af53a11090dd-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.791003 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.790907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr8w\" (UniqueName: \"kubernetes.io/projected/cf60079b-1b6e-42a5-bc0f-af53a11090dd-kube-api-access-8gr8w\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.791003 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.790985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf60079b-1b6e-42a5-bc0f-af53a11090dd-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.791082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.791022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf60079b-1b6e-42a5-bc0f-af53a11090dd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.791645 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.791621 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf60079b-1b6e-42a5-bc0f-af53a11090dd-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.793277 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.793255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf60079b-1b6e-42a5-bc0f-af53a11090dd-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.793353 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.793259 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf60079b-1b6e-42a5-bc0f-af53a11090dd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.799025 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.799003 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gr8w\" (UniqueName: \"kubernetes.io/projected/cf60079b-1b6e-42a5-bc0f-af53a11090dd-kube-api-access-8gr8w\") pod \"prometheus-operator-5945b5cfd-nfvp9\" (UID: \"cf60079b-1b6e-42a5-bc0f-af53a11090dd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:12.904555 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:12.904486 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" Jul 18 02:41:13.019736 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.019709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9"] Jul 18 02:41:13.022732 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:13.022708 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf60079b_1b6e_42a5_bc0f_af53a11090dd.slice/crio-6094f6ef1afd24b9ce3759a0b55fdb5345a19344cdb8a7058a568315922647e3 WatchSource:0}: Error finding container 6094f6ef1afd24b9ce3759a0b55fdb5345a19344cdb8a7058a568315922647e3: Status 404 returned error can't find the container with id 6094f6ef1afd24b9ce3759a0b55fdb5345a19344cdb8a7058a568315922647e3 Jul 18 02:41:13.673702 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.673663 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f6b96885f-c9mrq"] Jul 18 02:41:13.676010 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.675987 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.679774 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.679749 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 02:41:13.680424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.680399 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 18 02:41:13.680530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.680464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 02:41:13.680530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.680492 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 02:41:13.680530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.680515 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 02:41:13.680712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.680414 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-662v8\"" Jul 18 02:41:13.680712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.680652 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 18 02:41:13.681414 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.681396 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 02:41:13.688619 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.688594 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6b96885f-c9mrq"] Jul 18 02:41:13.798149 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.798117 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-service-ca\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.798309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.798185 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-serving-cert\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.798309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.798229 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-console-config\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.798430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.798358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-oauth-serving-cert\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.798430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.798402 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh8p4\" (UniqueName: \"kubernetes.io/projected/36d6c74a-1633-4959-ad46-713e150cfd17-kube-api-access-rh8p4\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.798529 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.798441 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-oauth-config\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.899677 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.899641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-service-ca\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.899902 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.899699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-serving-cert\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.899902 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.899722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-console-config\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.899902 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.899765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-oauth-serving-cert\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.899902 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.899785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh8p4\" (UniqueName: \"kubernetes.io/projected/36d6c74a-1633-4959-ad46-713e150cfd17-kube-api-access-rh8p4\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.899902 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.899825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-oauth-config\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.900494 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.900467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-service-ca\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.900603 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.900534 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-oauth-serving-cert\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.900603 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.900534 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-console-config\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.902590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.902566 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-serving-cert\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.902753 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.902735 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-oauth-config\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.912599 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.912573 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh8p4\" (UniqueName: \"kubernetes.io/projected/36d6c74a-1633-4959-ad46-713e150cfd17-kube-api-access-rh8p4\") pod \"console-7f6b96885f-c9mrq\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:13.988221 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:13.988159 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:14.026035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.025998 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" event={"ID":"cf60079b-1b6e-42a5-bc0f-af53a11090dd","Type":"ContainerStarted","Data":"6094f6ef1afd24b9ce3759a0b55fdb5345a19344cdb8a7058a568315922647e3"} Jul 18 02:41:14.392915 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.392892 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6b96885f-c9mrq"] Jul 18 02:41:14.396009 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:14.395978 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d6c74a_1633_4959_ad46_713e150cfd17.slice/crio-a415e8cf61ebd5af0cf515bf14a6d2b3a2c1776eb9606aa64e1dcbc1a3afcdd3 WatchSource:0}: Error finding container a415e8cf61ebd5af0cf515bf14a6d2b3a2c1776eb9606aa64e1dcbc1a3afcdd3: Status 404 returned error can't find the container with id a415e8cf61ebd5af0cf515bf14a6d2b3a2c1776eb9606aa64e1dcbc1a3afcdd3 Jul 18 02:41:14.401984 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.401945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:41:14.402060 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.402022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:41:14.404438 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.404417 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 02:41:14.404538 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.404420 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 02:41:14.414737 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.414722 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 02:41:14.414894 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.414873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e42b787c-d4c0-4bf0-bf67-bc10adb618ff-metrics-certs\") pod \"network-metrics-daemon-jzqvv\" (UID: \"e42b787c-d4c0-4bf0-bf67-bc10adb618ff\") " pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:41:14.425306 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.425283 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pbhz\" (UniqueName: \"kubernetes.io/projected/e22fbac2-5236-4ef7-801c-8ee05261b455-kube-api-access-6pbhz\") pod \"network-check-target-ltr2b\" (UID: \"e22fbac2-5236-4ef7-801c-8ee05261b455\") " pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:41:14.468750 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.468726 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zt6zk\"" Jul 18 02:41:14.476910 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.476889 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:41:14.483740 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.483724 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzcn2\"" Jul 18 02:41:14.491686 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.491662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jzqvv" Jul 18 02:41:14.502455 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.502429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:41:14.504959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.504929 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 02:41:14.515818 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.515765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba51b136-217a-436d-80ba-8b5332c264fd-original-pull-secret\") pod \"global-pull-secret-syncer-nswl4\" (UID: \"ba51b136-217a-436d-80ba-8b5332c264fd\") " pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:41:14.620499 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.620465 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ltr2b"] Jul 18 02:41:14.623650 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:14.623624 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22fbac2_5236_4ef7_801c_8ee05261b455.slice/crio-e4568878633fa92931851a238294c53d866105bf44aea69bd19daf4f4119959b WatchSource:0}: Error finding container e4568878633fa92931851a238294c53d866105bf44aea69bd19daf4f4119959b: Status 404 returned error can't find the container with id e4568878633fa92931851a238294c53d866105bf44aea69bd19daf4f4119959b Jul 18 02:41:14.637469 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.637450 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jzqvv"] Jul 18 02:41:14.639941 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:14.639918 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42b787c_d4c0_4bf0_bf67_bc10adb618ff.slice/crio-f6e28971859468478c8b0e99feb8ee0d0167fa216e8d294f4789a88df9549769 WatchSource:0}: Error finding container f6e28971859468478c8b0e99feb8ee0d0167fa216e8d294f4789a88df9549769: Status 404 returned error can't find the container with id f6e28971859468478c8b0e99feb8ee0d0167fa216e8d294f4789a88df9549769 Jul 18 02:41:14.775459 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.775438 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nswl4" Jul 18 02:41:14.893959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.893930 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nswl4"] Jul 18 02:41:14.897396 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:14.897373 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba51b136_217a_436d_80ba_8b5332c264fd.slice/crio-709fa43cde5bc5668ca24020f2ee221ac1ceb9a059e18f610df5277f135cd6b6 WatchSource:0}: Error finding container 709fa43cde5bc5668ca24020f2ee221ac1ceb9a059e18f610df5277f135cd6b6: Status 404 returned error can't find the container with id 709fa43cde5bc5668ca24020f2ee221ac1ceb9a059e18f610df5277f135cd6b6 Jul 18 02:41:14.906096 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.906073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:41:14.908340 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:14.908320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02c7cf7d-935f-4bcc-8905-bd040c739002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wbsjv\" (UID: \"02c7cf7d-935f-4bcc-8905-bd040c739002\") " pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:41:15.031388 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.031360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" event={"ID":"cf60079b-1b6e-42a5-bc0f-af53a11090dd","Type":"ContainerStarted","Data":"7020181cd91b53340b09bbd245e62ed75cb50a4f8303fdbe3a0cf756761e611d"} Jul 18 02:41:15.031771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.031420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" event={"ID":"cf60079b-1b6e-42a5-bc0f-af53a11090dd","Type":"ContainerStarted","Data":"86c44710638d0cfdd62eee7dedcf8a4fe1bb41e413c93522aec074c4097e9da5"} Jul 18 02:41:15.032791 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.032760 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nswl4" event={"ID":"ba51b136-217a-436d-80ba-8b5332c264fd","Type":"ContainerStarted","Data":"709fa43cde5bc5668ca24020f2ee221ac1ceb9a059e18f610df5277f135cd6b6"} Jul 18 02:41:15.034072 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.034047 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jzqvv" event={"ID":"e42b787c-d4c0-4bf0-bf67-bc10adb618ff","Type":"ContainerStarted","Data":"f6e28971859468478c8b0e99feb8ee0d0167fa216e8d294f4789a88df9549769"} Jul 18 02:41:15.035525 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.035503 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6b96885f-c9mrq" event={"ID":"36d6c74a-1633-4959-ad46-713e150cfd17","Type":"ContainerStarted","Data":"a415e8cf61ebd5af0cf515bf14a6d2b3a2c1776eb9606aa64e1dcbc1a3afcdd3"} Jul 18 02:41:15.036867 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.036846 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ltr2b" event={"ID":"e22fbac2-5236-4ef7-801c-8ee05261b455","Type":"ContainerStarted","Data":"e4568878633fa92931851a238294c53d866105bf44aea69bd19daf4f4119959b"} Jul 18 02:41:15.048687 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.048646 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nfvp9" podStartSLOduration=1.7759293839999999 podStartE2EDuration="3.048634482s" totalImagesPullingTime="1.272705098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:15.04820288 +0000 UTC m=+66.836976651" watchObservedRunningTime="2026-07-18 02:41:15.048634482 +0000 UTC m=+66.837408251" Jul 18 02:41:15.130932 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.130874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nv5k\"" Jul 18 02:41:15.138949 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.138929 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wbsjv" Jul 18 02:41:15.306382 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.306295 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wbsjv"] Jul 18 02:41:15.308826 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:15.308794 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c7cf7d_935f_4bcc_8905_bd040c739002.slice/crio-65f1b2ad657acd98731ea9580340acaf6a533ee45deb63b412a33de445329238 WatchSource:0}: Error finding container 65f1b2ad657acd98731ea9580340acaf6a533ee45deb63b412a33de445329238: Status 404 returned error can't find the container with id 65f1b2ad657acd98731ea9580340acaf6a533ee45deb63b412a33de445329238 Jul 18 02:41:15.648343 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.648312 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-698b58bd4-rzzd5"] Jul 18 02:41:15.665582 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.665537 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-698b58bd4-rzzd5"] Jul 18 02:41:15.665719 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.665652 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.673915 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.673718 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 18 02:41:15.711862 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.711830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-oauth-config\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.712029 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.711882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-oauth-serving-cert\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.712029 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.711951 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-config\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.712029 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.712002 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-trusted-ca-bundle\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.712204 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.712032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tfhn\" (UniqueName: \"kubernetes.io/projected/714fac21-19ce-4bb9-8fb2-afbf082bd20c-kube-api-access-9tfhn\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.712204 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.712114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-serving-cert\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.712204 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.712140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-service-ca\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813188 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-oauth-config\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813339 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-oauth-serving-cert\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813339 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-config\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813339 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-trusted-ca-bundle\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813511 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tfhn\" (UniqueName: \"kubernetes.io/projected/714fac21-19ce-4bb9-8fb2-afbf082bd20c-kube-api-access-9tfhn\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813511 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-serving-cert\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.813511 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.813464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-service-ca\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.814264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.814199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-config\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.814264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.814187 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-trusted-ca-bundle\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.814264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.814243 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-service-ca\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.814769 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.814743 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-oauth-serving-cert\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.816075 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.816054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-oauth-config\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.817585 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.817563 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-serving-cert\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.822991 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.822942 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tfhn\" (UniqueName: \"kubernetes.io/projected/714fac21-19ce-4bb9-8fb2-afbf082bd20c-kube-api-access-9tfhn\") pod \"console-698b58bd4-rzzd5\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:15.978785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:15.978709 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:16.042793 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:16.042701 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wbsjv" event={"ID":"02c7cf7d-935f-4bcc-8905-bd040c739002","Type":"ContainerStarted","Data":"65f1b2ad657acd98731ea9580340acaf6a533ee45deb63b412a33de445329238"} Jul 18 02:41:16.202433 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:16.202405 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-698b58bd4-rzzd5"] Jul 18 02:41:16.206670 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:16.206635 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714fac21_19ce_4bb9_8fb2_afbf082bd20c.slice/crio-65fcebe5583a743feae1bb393cb99545a033da3defd83cec59b4e49d4ef1fd27 WatchSource:0}: Error finding container 65fcebe5583a743feae1bb393cb99545a033da3defd83cec59b4e49d4ef1fd27: Status 404 returned error can't find the container with id 65fcebe5583a743feae1bb393cb99545a033da3defd83cec59b4e49d4ef1fd27 Jul 18 02:41:17.042194 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.041285 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kfqqj"] Jul 18 02:41:17.054833 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.054641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jzqvv" event={"ID":"e42b787c-d4c0-4bf0-bf67-bc10adb618ff","Type":"ContainerStarted","Data":"0402cda833da7bd445380c824e9b87f9bee91c2e46920ed3e5ea03694079b4e2"} Jul 18 02:41:17.054833 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.054686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jzqvv" event={"ID":"e42b787c-d4c0-4bf0-bf67-bc10adb618ff","Type":"ContainerStarted","Data":"2c47452f12184e11257c7f21419c4035ff485655f2e768c5e1786ba22ea9f0a2"} Jul 18 02:41:17.054833 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.054789 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.061519 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.060000 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 02:41:17.061519 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.060204 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 02:41:17.061519 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.061136 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 02:41:17.061519 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.061331 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2x9df\"" Jul 18 02:41:17.064733 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.064668 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698b58bd4-rzzd5" event={"ID":"714fac21-19ce-4bb9-8fb2-afbf082bd20c","Type":"ContainerStarted","Data":"65fcebe5583a743feae1bb393cb99545a033da3defd83cec59b4e49d4ef1fd27"} Jul 18 02:41:17.078933 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.078729 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jzqvv" podStartSLOduration=67.681424866 podStartE2EDuration="1m9.078712615s" totalImagesPullingTime="1.397287749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:17.07626006 +0000 UTC m=+68.865033831" watchObservedRunningTime="2026-07-18 02:41:17.078712615 +0000 UTC m=+68.867486385" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-wtmp\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132251 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-textfile\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132278 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppjjm\" (UniqueName: \"kubernetes.io/projected/56c07650-f10f-4901-ab0e-bf61796f6fe4-kube-api-access-ppjjm\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132310 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-sys\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132390 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-root\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132441 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56c07650-f10f-4901-ab0e-bf61796f6fe4-metrics-client-ca\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-tls\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.133708 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.132493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-accelerators-collector-config\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-root\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56c07650-f10f-4901-ab0e-bf61796f6fe4-metrics-client-ca\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-tls\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-accelerators-collector-config\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-wtmp\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233912 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-textfile\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppjjm\" (UniqueName: \"kubernetes.io/projected/56c07650-f10f-4901-ab0e-bf61796f6fe4-kube-api-access-ppjjm\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.233982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-sys\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.234071 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-sys\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.235036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-accelerators-collector-config\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.235099 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-root\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.235763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.235496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56c07650-f10f-4901-ab0e-bf61796f6fe4-metrics-client-ca\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.242435 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.236747 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-textfile\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.242435 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.236880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-wtmp\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.253611 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.253533 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-tls\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.260073 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.260031 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppjjm\" (UniqueName: \"kubernetes.io/projected/56c07650-f10f-4901-ab0e-bf61796f6fe4-kube-api-access-ppjjm\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.260417 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.260382 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56c07650-f10f-4901-ab0e-bf61796f6fe4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kfqqj\" (UID: \"56c07650-f10f-4901-ab0e-bf61796f6fe4\") " pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:17.377377 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:17.377070 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kfqqj" Jul 18 02:41:18.076790 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.076671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfqqj" event={"ID":"56c07650-f10f-4901-ab0e-bf61796f6fe4","Type":"ContainerStarted","Data":"b23071006ffca9c5cff0e81d5a511bccdd4296b6c8751a6a434a6a35dc15ce51"} Jul 18 02:41:18.089768 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.089733 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:41:18.111754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.111694 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:41:18.111983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.111887 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.115679 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.115654 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 18 02:41:18.116263 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.115890 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 18 02:41:18.116263 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.116119 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 18 02:41:18.116407 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.116326 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 18 02:41:18.117152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.116519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 18 02:41:18.117152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.116644 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 18 02:41:18.117152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.116877 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 18 02:41:18.117152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.116956 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6qklv\"" Jul 18 02:41:18.117152 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.117062 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 18 02:41:18.117439 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.117214 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 18 02:41:18.249836 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.249800 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.249849 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.249887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.249914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.249945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-web-config\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.249999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250035 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-config-volume\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250076 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2krz\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-kube-api-access-b2krz\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250105 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-config-out\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250206 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.250367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.250231 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351544 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351468 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351544 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351653 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-web-config\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.351803 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.352327 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351804 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-config-volume\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.352327 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2krz\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-kube-api-access-b2krz\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.352327 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351898 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.352327 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.351928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-config-out\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.353136 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.352961 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.353614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.353574 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.354527 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.354423 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.358038 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.358014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-config-out\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.359078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.359001 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.359078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.359049 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.359483 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.359463 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.359676 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.359633 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-config-volume\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.360271 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.360231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.361153 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.361130 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.361522 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.361499 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.362388 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.362341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-web-config\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.367928 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.367886 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2krz\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-kube-api-access-b2krz\") pod \"alertmanager-main-0\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:18.429363 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:18.429170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:41:21.306158 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.306112 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-89dc48964-lm9rd"] Jul 18 02:41:21.308447 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.308422 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.311906 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.311880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-m9sst\"" Jul 18 02:41:21.312066 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.311934 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 18 02:41:21.312066 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.311946 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 18 02:41:21.312066 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.312040 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1sn50khbn60nc\"" Jul 18 02:41:21.312246 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.311949 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 18 02:41:21.312246 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.311880 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 18 02:41:21.317274 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.317228 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-89dc48964-lm9rd"] Jul 18 02:41:21.378304 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378270 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-client-ca-bundle\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.378304 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-secret-metrics-server-tls\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.378485 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-secret-metrics-server-client-certs\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.378485 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df8zt\" (UniqueName: \"kubernetes.io/projected/fe81a410-7b19-4517-bb67-548da9268822-kube-api-access-df8zt\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.378485 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fe81a410-7b19-4517-bb67-548da9268822-metrics-server-audit-profiles\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.378612 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe81a410-7b19-4517-bb67-548da9268822-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.378612 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.378557 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fe81a410-7b19-4517-bb67-548da9268822-audit-log\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.479626 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-client-ca-bundle\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.479786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-secret-metrics-server-tls\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.479786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-secret-metrics-server-client-certs\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.479786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df8zt\" (UniqueName: \"kubernetes.io/projected/fe81a410-7b19-4517-bb67-548da9268822-kube-api-access-df8zt\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.479786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fe81a410-7b19-4517-bb67-548da9268822-metrics-server-audit-profiles\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.480078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe81a410-7b19-4517-bb67-548da9268822-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.480078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.479847 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fe81a410-7b19-4517-bb67-548da9268822-audit-log\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.480264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.480233 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fe81a410-7b19-4517-bb67-548da9268822-audit-log\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.480893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.480850 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe81a410-7b19-4517-bb67-548da9268822-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.481135 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.481113 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fe81a410-7b19-4517-bb67-548da9268822-metrics-server-audit-profiles\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.482459 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.482437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-client-ca-bundle\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.482635 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.482610 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-secret-metrics-server-tls\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.482802 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.482776 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fe81a410-7b19-4517-bb67-548da9268822-secret-metrics-server-client-certs\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.488661 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.488636 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df8zt\" (UniqueName: \"kubernetes.io/projected/fe81a410-7b19-4517-bb67-548da9268822-kube-api-access-df8zt\") pod \"metrics-server-89dc48964-lm9rd\" (UID: \"fe81a410-7b19-4517-bb67-548da9268822\") " pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.620736 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.620612 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:21.762490 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.762424 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r"] Jul 18 02:41:21.764721 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.764701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:21.767628 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.766940 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 18 02:41:21.767628 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.767218 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hmcdm\"" Jul 18 02:41:21.774300 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.774276 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r"] Jul 18 02:41:21.883246 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.883163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56dkp\" (UniqueName: \"kubernetes.io/projected/25336ac5-f4c9-49f8-92a4-d5c93a5686da-kube-api-access-56dkp\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:21.883246 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.883215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:21.984128 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.984095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56dkp\" (UniqueName: \"kubernetes.io/projected/25336ac5-f4c9-49f8-92a4-d5c93a5686da-kube-api-access-56dkp\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:21.984317 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.984148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:21.984317 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:41:21.984266 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 18 02:41:21.984435 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:41:21.984336 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert podName:25336ac5-f4c9-49f8-92a4-d5c93a5686da nodeName:}" failed. No retries permitted until 2026-07-18 02:41:22.484316216 +0000 UTC m=+74.273089964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-k5p2r" (UID: "25336ac5-f4c9-49f8-92a4-d5c93a5686da") : secret "monitoring-plugin-cert" not found Jul 18 02:41:21.996324 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:21.996292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56dkp\" (UniqueName: \"kubernetes.io/projected/25336ac5-f4c9-49f8-92a4-d5c93a5686da-kube-api-access-56dkp\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:22.229595 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.229510 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7595c577bc-pxcq6"] Jul 18 02:41:22.233771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.233739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.236268 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.236238 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6mlz7\"" Jul 18 02:41:22.236416 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.236366 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 18 02:41:22.236416 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.236375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 18 02:41:22.236526 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.236489 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 18 02:41:22.236580 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.236556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 18 02:41:22.236920 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.236884 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 18 02:41:22.241656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.241638 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 18 02:41:22.244893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.244855 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7595c577bc-pxcq6"] Jul 18 02:41:22.388076 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-federate-client-tls\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388083 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-metrics-client-ca\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j6l6\" (UniqueName: \"kubernetes.io/projected/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-kube-api-access-6j6l6\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-secret-telemeter-client\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388254 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-telemeter-client-tls\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388293 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388419 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-serving-certs-ca-bundle\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.388512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.388480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.489396 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.489318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:22.489396 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.489377 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-serving-certs-ca-bundle\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.489591 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:41:22.489482 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 18 02:41:22.489591 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:41:22.489554 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert podName:25336ac5-f4c9-49f8-92a4-d5c93a5686da nodeName:}" failed. No retries permitted until 2026-07-18 02:41:23.489534113 +0000 UTC m=+75.278307866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-k5p2r" (UID: "25336ac5-f4c9-49f8-92a4-d5c93a5686da") : secret "monitoring-plugin-cert" not found Jul 18 02:41:22.490062 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.489489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490062 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-federate-client-tls\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490062 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-metrics-client-ca\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j6l6\" (UniqueName: \"kubernetes.io/projected/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-kube-api-access-6j6l6\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-serving-certs-ca-bundle\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-secret-telemeter-client\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-telemeter-client-tls\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.490316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.490209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.491084 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.491032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-metrics-client-ca\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.493141 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.493024 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.493141 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.493100 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-federate-client-tls\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.493293 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.493149 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-secret-telemeter-client\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.493351 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.493290 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-telemeter-client-tls\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.500864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.500823 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j6l6\" (UniqueName: \"kubernetes.io/projected/fc3fb636-0a27-43fc-b8a1-757ac248cbcb-kube-api-access-6j6l6\") pod \"telemeter-client-7595c577bc-pxcq6\" (UID: \"fc3fb636-0a27-43fc-b8a1-757ac248cbcb\") " pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:22.546825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:22.546799 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" Jul 18 02:41:23.458914 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.458873 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f6b96885f-c9mrq"] Jul 18 02:41:23.490093 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.490059 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66d494d646-v9pvl"] Jul 18 02:41:23.501305 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.501280 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d494d646-v9pvl"] Jul 18 02:41:23.501444 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.501391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.501517 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.501452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:23.504381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.504332 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25336ac5-f4c9-49f8-92a4-d5c93a5686da-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k5p2r\" (UID: \"25336ac5-f4c9-49f8-92a4-d5c93a5686da\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:23.579633 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.579350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:23.602613 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-oauth-config\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.602763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602627 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wxqw\" (UniqueName: \"kubernetes.io/projected/e4da3129-84cd-40b9-8c87-8ee558ca17bb-kube-api-access-6wxqw\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.602763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602666 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-oauth-serving-cert\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.602763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-serving-cert\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.602763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-config\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.603004 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-trusted-ca-bundle\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.603004 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.602847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-service-ca\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.703658 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-oauth-serving-cert\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.703827 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703666 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-serving-cert\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.703827 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-config\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.703827 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703721 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-trusted-ca-bundle\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.703827 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703773 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-service-ca\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.703827 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-oauth-config\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.704102 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.703851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wxqw\" (UniqueName: \"kubernetes.io/projected/e4da3129-84cd-40b9-8c87-8ee558ca17bb-kube-api-access-6wxqw\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.704499 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.704437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-oauth-serving-cert\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.704499 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.704485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-service-ca\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.704722 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.704699 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-trusted-ca-bundle\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.704851 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.704816 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-config\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.706749 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.706727 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-oauth-config\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.706837 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.706759 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-serving-cert\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.712280 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.712228 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wxqw\" (UniqueName: \"kubernetes.io/projected/e4da3129-84cd-40b9-8c87-8ee558ca17bb-kube-api-access-6wxqw\") pod \"console-66d494d646-v9pvl\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:23.820728 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:23.820698 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:25.371919 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:25.370769 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:41:25.389872 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:25.389368 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r"] Jul 18 02:41:25.431271 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:25.430950 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d494d646-v9pvl"] Jul 18 02:41:25.440736 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:25.440703 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4da3129_84cd_40b9_8c87_8ee558ca17bb.slice/crio-2096e0c713dc4263bb85796f8a5c5798e264826f78297d295ec9b6f763084ece WatchSource:0}: Error finding container 2096e0c713dc4263bb85796f8a5c5798e264826f78297d295ec9b6f763084ece: Status 404 returned error can't find the container with id 2096e0c713dc4263bb85796f8a5c5798e264826f78297d295ec9b6f763084ece Jul 18 02:41:25.472463 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:25.472392 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-89dc48964-lm9rd"] Jul 18 02:41:25.475962 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:25.475934 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7595c577bc-pxcq6"] Jul 18 02:41:25.476665 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:25.476630 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe81a410_7b19_4517_bb67_548da9268822.slice/crio-b1f014a3479d672d301e9f662de5621614366211e040ad1d0dd2d27eb3e91c49 WatchSource:0}: Error finding container b1f014a3479d672d301e9f662de5621614366211e040ad1d0dd2d27eb3e91c49: Status 404 returned error can't find the container with id b1f014a3479d672d301e9f662de5621614366211e040ad1d0dd2d27eb3e91c49 Jul 18 02:41:26.123878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.123840 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698b58bd4-rzzd5" event={"ID":"714fac21-19ce-4bb9-8fb2-afbf082bd20c","Type":"ContainerStarted","Data":"b06cc9b40cd242ceb078ff762f2c38c13ee09b46401d6855ec35a187a4883719"} Jul 18 02:41:26.126501 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.126472 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" event={"ID":"fc3fb636-0a27-43fc-b8a1-757ac248cbcb","Type":"ContainerStarted","Data":"05adc49ca8bdc0d6c9200688fe65aa3854cc0b6b8f0de774c8ca4a258d69dcc9"} Jul 18 02:41:26.129187 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.129024 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"15d67475d73e16158acc8cbf64765bda6a08837e4385177f58885ab1164354b2"} Jul 18 02:41:26.135060 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.134357 2560 generic.go:356] "Generic (PLEG): container finished" podID="56c07650-f10f-4901-ab0e-bf61796f6fe4" containerID="25d78ee77da298da9cee54384d8a3afb85f49d55ac57877bb86a47c3c0c404a2" exitCode=0 Jul 18 02:41:26.135060 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.134433 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfqqj" event={"ID":"56c07650-f10f-4901-ab0e-bf61796f6fe4","Type":"ContainerDied","Data":"25d78ee77da298da9cee54384d8a3afb85f49d55ac57877bb86a47c3c0c404a2"} Jul 18 02:41:26.141303 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.141178 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d494d646-v9pvl" event={"ID":"e4da3129-84cd-40b9-8c87-8ee558ca17bb","Type":"ContainerStarted","Data":"a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6"} Jul 18 02:41:26.141303 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.141258 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d494d646-v9pvl" event={"ID":"e4da3129-84cd-40b9-8c87-8ee558ca17bb","Type":"ContainerStarted","Data":"2096e0c713dc4263bb85796f8a5c5798e264826f78297d295ec9b6f763084ece"} Jul 18 02:41:26.142804 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.142757 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-698b58bd4-rzzd5" podStartSLOduration=3.055690722 podStartE2EDuration="11.142741013s" totalImagesPullingTime="8.087050291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:26.14127109 +0000 UTC m=+77.930044858" watchObservedRunningTime="2026-07-18 02:41:26.142741013 +0000 UTC m=+77.931514784" Jul 18 02:41:26.150134 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.150104 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" event={"ID":"25336ac5-f4c9-49f8-92a4-d5c93a5686da","Type":"ContainerStarted","Data":"ce2127fc4b5bf799815157db46a0507c3ba2a69d022fbff3467b235b8405c6d3"} Jul 18 02:41:26.153101 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.153073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" event={"ID":"fe81a410-7b19-4517-bb67-548da9268822","Type":"ContainerStarted","Data":"b1f014a3479d672d301e9f662de5621614366211e040ad1d0dd2d27eb3e91c49"} Jul 18 02:41:26.160833 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.160268 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nswl4" event={"ID":"ba51b136-217a-436d-80ba-8b5332c264fd","Type":"ContainerStarted","Data":"fd05853f5113e19936f3f11cdcc784d166f1c01fec64dddc10725bf82349256b"} Jul 18 02:41:26.165586 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.165560 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6b96885f-c9mrq" event={"ID":"36d6c74a-1633-4959-ad46-713e150cfd17","Type":"ContainerStarted","Data":"cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673"} Jul 18 02:41:26.177267 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.175675 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nswl4" podStartSLOduration=67.959446747 podStartE2EDuration="1m18.175658553s" totalImagesPullingTime="10.216211806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:26.175478514 +0000 UTC m=+77.964252285" watchObservedRunningTime="2026-07-18 02:41:26.175658553 +0000 UTC m=+77.964432316" Jul 18 02:41:26.180106 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.179851 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wbsjv" event={"ID":"02c7cf7d-935f-4bcc-8905-bd040c739002","Type":"ContainerStarted","Data":"17f8da6c6e2e00f924225521c19db71e6065e242fe6de52ec4af9f388ca91cc1"} Jul 18 02:41:26.184141 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.184061 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ltr2b" event={"ID":"e22fbac2-5236-4ef7-801c-8ee05261b455","Type":"ContainerStarted","Data":"e8ee5980db203f07c65067a2cf1a47706f3aa0d86cea61fd9686f682627ae491"} Jul 18 02:41:26.185601 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.184528 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:41:26.193826 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.192671 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66d494d646-v9pvl" podStartSLOduration=3.19265515 podStartE2EDuration="3.19265515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:26.191820508 +0000 UTC m=+77.980594280" watchObservedRunningTime="2026-07-18 02:41:26.19265515 +0000 UTC m=+77.981428921" Jul 18 02:41:26.229453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:26.226997 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f6b96885f-c9mrq" podStartSLOduration=3.324875952 podStartE2EDuration="13.226963355s" totalImagesPullingTime="9.902087403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:26.209627562 +0000 UTC m=+77.998401334" watchObservedRunningTime="2026-07-18 02:41:26.226963355 +0000 UTC m=+78.015737126" Jul 18 02:41:27.192204 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:27.192160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfqqj" event={"ID":"56c07650-f10f-4901-ab0e-bf61796f6fe4","Type":"ContainerStarted","Data":"fc729a211a66a5898d2e66bb6d925ff23c37c667c587fd703abe1c8f1f574da6"} Jul 18 02:41:27.192678 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:27.192276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfqqj" event={"ID":"56c07650-f10f-4901-ab0e-bf61796f6fe4","Type":"ContainerStarted","Data":"ef8092260bb34382e1a91b03f6ccbf51102150bfb301aefb04e865f5853c3a8c"} Jul 18 02:41:27.213899 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:27.213845 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ltr2b" podStartSLOduration=69.539156046 podStartE2EDuration="1m19.213827403s" totalImagesPullingTime="9.674671357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:26.228810685 +0000 UTC m=+78.017584456" watchObservedRunningTime="2026-07-18 02:41:27.213827403 +0000 UTC m=+79.002601175" Jul 18 02:41:27.214507 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:27.214467 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kfqqj" podStartSLOduration=2.364669052 podStartE2EDuration="10.214458558s" totalImagesPullingTime="7.750136504s" totalInitContainerRuntime="99.653002ms" isStatefulPod=true podCreationTimestamp="2026-07-18 02:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:27.212368836 +0000 UTC m=+79.001142608" watchObservedRunningTime="2026-07-18 02:41:27.214458558 +0000 UTC m=+79.003232328" Jul 18 02:41:29.013886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:29.013851 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b9fb5fd8d-pmrlx" Jul 18 02:41:31.211645 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.211606 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" event={"ID":"25336ac5-f4c9-49f8-92a4-d5c93a5686da","Type":"ContainerStarted","Data":"79681376ef765600f173178a6b971dce053200cb38e48faef886f9bb3dcb9d42"} Jul 18 02:41:31.212124 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.212000 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:31.213588 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.213497 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" event={"ID":"fe81a410-7b19-4517-bb67-548da9268822","Type":"ContainerStarted","Data":"d04822ab005178de5430e8462bd7fa883d2b280bf81f39e2d1eff4432e9b0133"} Jul 18 02:41:31.215943 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.215900 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wbsjv" event={"ID":"02c7cf7d-935f-4bcc-8905-bd040c739002","Type":"ContainerStarted","Data":"4d381552e78045e3567ae582932159d6ad2a594755cab414b5b87956fca01e39"} Jul 18 02:41:31.217641 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.217619 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" Jul 18 02:41:31.219156 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.219130 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" event={"ID":"fc3fb636-0a27-43fc-b8a1-757ac248cbcb","Type":"ContainerStarted","Data":"fc127452d574c5c28fffee0f4b758e8e3268fa08d22224308cdbb94d74de8dad"} Jul 18 02:41:31.219272 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.219184 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" event={"ID":"fc3fb636-0a27-43fc-b8a1-757ac248cbcb","Type":"ContainerStarted","Data":"4a942d7e9f3d8be2da16e70655f55e9e28e8234a99a457c92423a70879efe396"} Jul 18 02:41:31.219272 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.219213 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" event={"ID":"fc3fb636-0a27-43fc-b8a1-757ac248cbcb","Type":"ContainerStarted","Data":"4616d69aa510f6c2030bd1e1660c3b16beb0bf09566016c6cab5a7d92fcc38ab"} Jul 18 02:41:31.220953 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.220922 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="bf01ab26572419a2447fded50addfaf381239b63c7db29e7bd8b7c4749f33f2c" exitCode=0 Jul 18 02:41:31.221056 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.220998 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"bf01ab26572419a2447fded50addfaf381239b63c7db29e7bd8b7c4749f33f2c"} Jul 18 02:41:31.228113 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.228075 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k5p2r" podStartSLOduration=5.370007079 podStartE2EDuration="10.228058207s" totalImagesPullingTime="4.858051128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:31.227382327 +0000 UTC m=+83.016156099" watchObservedRunningTime="2026-07-18 02:41:31.228058207 +0000 UTC m=+83.016831979" Jul 18 02:41:31.285034 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.284592 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" podStartSLOduration=5.491149415 podStartE2EDuration="10.284574776s" totalImagesPullingTime="4.793425361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:31.284053057 +0000 UTC m=+83.072826827" watchObservedRunningTime="2026-07-18 02:41:31.284574776 +0000 UTC m=+83.073348547" Jul 18 02:41:31.313256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:31.313206 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7595c577bc-pxcq6" podStartSLOduration=4.527559252 podStartE2EDuration="9.313191675s" totalImagesPullingTime="4.785632423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:31.311860974 +0000 UTC m=+83.100634745" watchObservedRunningTime="2026-07-18 02:41:31.313191675 +0000 UTC m=+83.101965444" Jul 18 02:41:32.073430 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.069607 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-698b58bd4-rzzd5"] Jul 18 02:41:32.103387 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.103353 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cdd576dd4-flnt4"] Jul 18 02:41:32.107536 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.107113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.117574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.117547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdd576dd4-flnt4"] Jul 18 02:41:32.289125 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-oauth-config\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.289576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj87z\" (UniqueName: \"kubernetes.io/projected/7aa45a40-196e-4477-89a7-3328b87370bc-kube-api-access-lj87z\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.289576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289272 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-service-ca\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.289576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-serving-cert\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.289576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-trusted-ca-bundle\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.289576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289412 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-console-config\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.289576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.289480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-oauth-serving-cert\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.390921 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.390831 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj87z\" (UniqueName: \"kubernetes.io/projected/7aa45a40-196e-4477-89a7-3328b87370bc-kube-api-access-lj87z\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.391149 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-service-ca\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.391267 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-serving-cert\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.391322 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-trusted-ca-bundle\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.391402 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-console-config\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.391460 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-oauth-serving-cert\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.391653 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391523 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-oauth-config\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.392372 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.391917 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-service-ca\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.392492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.392404 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-trusted-ca-bundle\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.392572 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.392543 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-console-config\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.392751 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.392729 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-oauth-serving-cert\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.395318 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.395263 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-oauth-config\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.399078 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.397051 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-serving-cert\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.409561 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.407554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj87z\" (UniqueName: \"kubernetes.io/projected/7aa45a40-196e-4477-89a7-3328b87370bc-kube-api-access-lj87z\") pod \"console-7cdd576dd4-flnt4\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:32.429562 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:32.429534 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:33.821896 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:33.821847 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:33.822330 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:33.821940 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:33.827576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:33.827554 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:33.989116 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:33.989074 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:34.242268 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:34.242192 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:41:34.612121 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:34.612099 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdd576dd4-flnt4"] Jul 18 02:41:34.628102 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:41:34.628076 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa45a40_196e_4477_89a7_3328b87370bc.slice/crio-fc170cd84e79c821203341e73f9b0d05c203c2ae3cf922b2e688c6eafaa41c24 WatchSource:0}: Error finding container fc170cd84e79c821203341e73f9b0d05c203c2ae3cf922b2e688c6eafaa41c24: Status 404 returned error can't find the container with id fc170cd84e79c821203341e73f9b0d05c203c2ae3cf922b2e688c6eafaa41c24 Jul 18 02:41:35.246851 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.246807 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wbsjv" event={"ID":"02c7cf7d-935f-4bcc-8905-bd040c739002","Type":"ContainerStarted","Data":"960ae842bb1491238df333cc6395fb2ade48cc16fcfe41da8ed0d8f353ead0ba"} Jul 18 02:41:35.248453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.248423 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd576dd4-flnt4" event={"ID":"7aa45a40-196e-4477-89a7-3328b87370bc","Type":"ContainerStarted","Data":"a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a"} Jul 18 02:41:35.248588 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.248465 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd576dd4-flnt4" event={"ID":"7aa45a40-196e-4477-89a7-3328b87370bc","Type":"ContainerStarted","Data":"fc170cd84e79c821203341e73f9b0d05c203c2ae3cf922b2e688c6eafaa41c24"} Jul 18 02:41:35.251502 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.251480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"8673ce649460347b058cb8012613620c84ae2c793f4eef8185799aded3455dfe"} Jul 18 02:41:35.251596 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.251551 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"4eecb1d5751271805ba22a2bf67bd3634e137785b108d534bb5b28e0678ede8c"} Jul 18 02:41:35.251596 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.251577 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"7602477253d80fddff171b0716f4cde381e3fdbe0474889b5a2b5fc533778a0f"} Jul 18 02:41:35.251670 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.251605 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"973e2be3cb678102853c5140ab968dbfa275a2172c703d64014a16146180e149"} Jul 18 02:41:35.251670 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.251629 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"155c1d4dee4a29f6b4d44634c00ef4d745583f4b2305407bbd7ebf20f717f5c5"} Jul 18 02:41:35.265682 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.265637 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wbsjv" podStartSLOduration=34.400128872 podStartE2EDuration="53.26562494s" totalImagesPullingTime="18.865496068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:40:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:35.263868575 +0000 UTC m=+87.052642346" watchObservedRunningTime="2026-07-18 02:41:35.26562494 +0000 UTC m=+87.054398709" Jul 18 02:41:35.280824 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.280786 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cdd576dd4-flnt4" podStartSLOduration=3.280774945 podStartE2EDuration="3.280774945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:35.280222056 +0000 UTC m=+87.068995827" watchObservedRunningTime="2026-07-18 02:41:35.280774945 +0000 UTC m=+87.069548714" Jul 18 02:41:35.979453 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:35.979371 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:36.258108 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:36.258007 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerStarted","Data":"0a87fb5c9c3c8b179fb78df8d4466b2bcf92fe9c380f86ecfc688c21d5abda0c"} Jul 18 02:41:36.285574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:36.285524 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.004120731 podStartE2EDuration="18.285508949s" totalImagesPullingTime="8.262615384s" totalInitContainerRuntime="18.772834ms" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:36.28292876 +0000 UTC m=+88.071702531" watchObservedRunningTime="2026-07-18 02:41:36.285508949 +0000 UTC m=+88.074282718" Jul 18 02:41:41.621645 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:41.621604 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:41.622177 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:41.622161 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:41:42.430252 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:42.430221 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:42.430414 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:42.430284 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:42.435118 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:42.435097 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:43.288774 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:43.288742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:41:43.338871 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:43.338832 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d494d646-v9pvl"] Jul 18 02:41:46.234900 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:46.234873 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9dw4n_a38f5409-3fa9-4a9e-9ada-4371a96c516e/serve-healthcheck-canary/0.log" Jul 18 02:41:51.207653 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.207588 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f6b96885f-c9mrq" podUID="36d6c74a-1633-4959-ad46-713e150cfd17" containerName="console" containerID="cri-o://cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673" gracePeriod=15 Jul 18 02:41:51.473174 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.473124 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:51.537292 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537253 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-oauth-serving-cert\") pod \"36d6c74a-1633-4959-ad46-713e150cfd17\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " Jul 18 02:41:51.537418 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537299 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-service-ca\") pod \"36d6c74a-1633-4959-ad46-713e150cfd17\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " Jul 18 02:41:51.537418 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537328 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh8p4\" (UniqueName: \"kubernetes.io/projected/36d6c74a-1633-4959-ad46-713e150cfd17-kube-api-access-rh8p4\") pod \"36d6c74a-1633-4959-ad46-713e150cfd17\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " Jul 18 02:41:51.537418 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537359 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-oauth-config\") pod \"36d6c74a-1633-4959-ad46-713e150cfd17\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " Jul 18 02:41:51.537591 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537541 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-console-config\") pod \"36d6c74a-1633-4959-ad46-713e150cfd17\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " Jul 18 02:41:51.537591 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537573 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-serving-cert\") pod \"36d6c74a-1633-4959-ad46-713e150cfd17\" (UID: \"36d6c74a-1633-4959-ad46-713e150cfd17\") " Jul 18 02:41:51.537692 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537619 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36d6c74a-1633-4959-ad46-713e150cfd17" (UID: "36d6c74a-1633-4959-ad46-713e150cfd17"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:51.537747 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537731 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-service-ca" (OuterVolumeSpecName: "service-ca") pod "36d6c74a-1633-4959-ad46-713e150cfd17" (UID: "36d6c74a-1633-4959-ad46-713e150cfd17"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:51.537878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537855 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-oauth-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:51.537878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537878 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-service-ca\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:51.538075 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.537951 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-console-config" (OuterVolumeSpecName: "console-config") pod "36d6c74a-1633-4959-ad46-713e150cfd17" (UID: "36d6c74a-1633-4959-ad46-713e150cfd17"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:51.539676 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.539630 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36d6c74a-1633-4959-ad46-713e150cfd17" (UID: "36d6c74a-1633-4959-ad46-713e150cfd17"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:41:51.539754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.539677 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d6c74a-1633-4959-ad46-713e150cfd17-kube-api-access-rh8p4" (OuterVolumeSpecName: "kube-api-access-rh8p4") pod "36d6c74a-1633-4959-ad46-713e150cfd17" (UID: "36d6c74a-1633-4959-ad46-713e150cfd17"). InnerVolumeSpecName "kube-api-access-rh8p4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:41:51.539754 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.539684 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36d6c74a-1633-4959-ad46-713e150cfd17" (UID: "36d6c74a-1633-4959-ad46-713e150cfd17"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:41:51.638234 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.638212 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh8p4\" (UniqueName: \"kubernetes.io/projected/36d6c74a-1633-4959-ad46-713e150cfd17-kube-api-access-rh8p4\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:51.638234 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.638233 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-oauth-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:51.638348 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.638243 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36d6c74a-1633-4959-ad46-713e150cfd17-console-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:51.638348 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:51.638252 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36d6c74a-1633-4959-ad46-713e150cfd17-console-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:52.319171 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.319141 2560 generic.go:356] "Generic (PLEG): container finished" podID="36d6c74a-1633-4959-ad46-713e150cfd17" containerID="cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673" exitCode=0 Jul 18 02:41:52.319574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.319184 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6b96885f-c9mrq" Jul 18 02:41:52.319574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.319236 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6b96885f-c9mrq" event={"ID":"36d6c74a-1633-4959-ad46-713e150cfd17","Type":"ContainerDied","Data":"cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673"} Jul 18 02:41:52.319574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.319289 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6b96885f-c9mrq" event={"ID":"36d6c74a-1633-4959-ad46-713e150cfd17","Type":"ContainerDied","Data":"a415e8cf61ebd5af0cf515bf14a6d2b3a2c1776eb9606aa64e1dcbc1a3afcdd3"} Jul 18 02:41:52.319574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.319311 2560 scope.go:117] "RemoveContainer" containerID="cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673" Jul 18 02:41:52.328685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.328665 2560 scope.go:117] "RemoveContainer" containerID="cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673" Jul 18 02:41:52.328946 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:41:52.328927 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673\": container with ID starting with cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673 not found: ID does not exist" containerID="cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673" Jul 18 02:41:52.329030 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.328955 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673"} err="failed to get container status \"cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673\": rpc error: code = NotFound desc = could not find container \"cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673\": container with ID starting with cf2af2e2c158056dec8055636bb0dd49d66db25595c2c2ae680b437937bdf673 not found: ID does not exist" Jul 18 02:41:52.339450 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.339427 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f6b96885f-c9mrq"] Jul 18 02:41:52.342912 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.342893 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f6b96885f-c9mrq"] Jul 18 02:41:52.755855 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:52.755775 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36d6c74a-1633-4959-ad46-713e150cfd17" path="/var/lib/kubelet/pods/36d6c74a-1633-4959-ad46-713e150cfd17/volumes" Jul 18 02:41:57.103515 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.103470 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-698b58bd4-rzzd5" podUID="714fac21-19ce-4bb9-8fb2-afbf082bd20c" containerName="console" containerID="cri-o://b06cc9b40cd242ceb078ff762f2c38c13ee09b46401d6855ec35a187a4883719" gracePeriod=15 Jul 18 02:41:57.342717 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.342686 2560 generic.go:356] "Generic (PLEG): container finished" podID="714fac21-19ce-4bb9-8fb2-afbf082bd20c" containerID="b06cc9b40cd242ceb078ff762f2c38c13ee09b46401d6855ec35a187a4883719" exitCode=0 Jul 18 02:41:57.342883 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.342757 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698b58bd4-rzzd5" event={"ID":"714fac21-19ce-4bb9-8fb2-afbf082bd20c","Type":"ContainerDied","Data":"b06cc9b40cd242ceb078ff762f2c38c13ee09b46401d6855ec35a187a4883719"} Jul 18 02:41:57.924989 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.921775 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:57.985467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985390 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tfhn\" (UniqueName: \"kubernetes.io/projected/714fac21-19ce-4bb9-8fb2-afbf082bd20c-kube-api-access-9tfhn\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985448 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-config\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985654 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985483 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-oauth-serving-cert\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985654 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985534 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-serving-cert\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985654 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985588 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-trusted-ca-bundle\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985654 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985607 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-oauth-config\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985654 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985638 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-service-ca\") pod \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\" (UID: \"714fac21-19ce-4bb9-8fb2-afbf082bd20c\") " Jul 18 02:41:57.985904 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985852 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-config" (OuterVolumeSpecName: "console-config") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:57.985904 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985865 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:57.986055 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.985929 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:57.986245 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.986219 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-service-ca" (OuterVolumeSpecName: "service-ca") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:41:57.988176 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.988145 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:41:57.988176 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.988145 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/714fac21-19ce-4bb9-8fb2-afbf082bd20c-kube-api-access-9tfhn" (OuterVolumeSpecName: "kube-api-access-9tfhn") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "kube-api-access-9tfhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:41:57.988312 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:57.988278 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "714fac21-19ce-4bb9-8fb2-afbf082bd20c" (UID: "714fac21-19ce-4bb9-8fb2-afbf082bd20c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:41:58.086752 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086721 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.086752 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086753 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-trusted-ca-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.086935 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086763 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-oauth-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.086935 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086773 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-service-ca\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.086935 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086782 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tfhn\" (UniqueName: \"kubernetes.io/projected/714fac21-19ce-4bb9-8fb2-afbf082bd20c-kube-api-access-9tfhn\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.086935 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086789 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-console-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.086935 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.086797 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714fac21-19ce-4bb9-8fb2-afbf082bd20c-oauth-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:41:58.200576 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.199686 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ltr2b" Jul 18 02:41:58.349444 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.349410 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698b58bd4-rzzd5" event={"ID":"714fac21-19ce-4bb9-8fb2-afbf082bd20c","Type":"ContainerDied","Data":"65fcebe5583a743feae1bb393cb99545a033da3defd83cec59b4e49d4ef1fd27"} Jul 18 02:41:58.349747 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.349726 2560 scope.go:117] "RemoveContainer" containerID="b06cc9b40cd242ceb078ff762f2c38c13ee09b46401d6855ec35a187a4883719" Jul 18 02:41:58.349884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.349533 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698b58bd4-rzzd5" Jul 18 02:41:58.377082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.377059 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-698b58bd4-rzzd5"] Jul 18 02:41:58.382138 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.382114 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-698b58bd4-rzzd5"] Jul 18 02:41:58.756297 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:41:58.756217 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="714fac21-19ce-4bb9-8fb2-afbf082bd20c" path="/var/lib/kubelet/pods/714fac21-19ce-4bb9-8fb2-afbf082bd20c/volumes" Jul 18 02:42:01.626498 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:01.626471 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:42:01.630188 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:01.630164 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-89dc48964-lm9rd" Jul 18 02:42:08.362661 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.362615 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66d494d646-v9pvl" podUID="e4da3129-84cd-40b9-8c87-8ee558ca17bb" containerName="console" containerID="cri-o://a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6" gracePeriod=15 Jul 18 02:42:08.602632 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.602612 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:42:08.663145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663089 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-oauth-config\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663148 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-oauth-serving-cert\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663166 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-trusted-ca-bundle\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663188 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-service-ca\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663427 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663273 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-serving-cert\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663427 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663331 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wxqw\" (UniqueName: \"kubernetes.io/projected/e4da3129-84cd-40b9-8c87-8ee558ca17bb-kube-api-access-6wxqw\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663427 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663371 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-config\") pod \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\" (UID: \"e4da3129-84cd-40b9-8c87-8ee558ca17bb\") " Jul 18 02:42:08.663588 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663541 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:42:08.663588 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663559 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-service-ca" (OuterVolumeSpecName: "service-ca") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:42:08.663684 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663637 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-oauth-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:08.663684 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663653 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-service-ca\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:08.663878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663849 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-config" (OuterVolumeSpecName: "console-config") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:42:08.663878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.663859 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:42:08.665374 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.665353 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:08.665573 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.665553 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4da3129-84cd-40b9-8c87-8ee558ca17bb-kube-api-access-6wxqw" (OuterVolumeSpecName: "kube-api-access-6wxqw") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "kube-api-access-6wxqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:42:08.665652 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.665634 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e4da3129-84cd-40b9-8c87-8ee558ca17bb" (UID: "e4da3129-84cd-40b9-8c87-8ee558ca17bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:08.763944 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.763925 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-oauth-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:08.764057 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.763946 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-trusted-ca-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:08.764057 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.763955 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:08.764057 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.763982 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wxqw\" (UniqueName: \"kubernetes.io/projected/e4da3129-84cd-40b9-8c87-8ee558ca17bb-kube-api-access-6wxqw\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:08.764057 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:08.763992 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4da3129-84cd-40b9-8c87-8ee558ca17bb-console-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:09.392408 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.392375 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4da3129-84cd-40b9-8c87-8ee558ca17bb" containerID="a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6" exitCode=0 Jul 18 02:42:09.392785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.392417 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d494d646-v9pvl" Jul 18 02:42:09.392785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.392456 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d494d646-v9pvl" event={"ID":"e4da3129-84cd-40b9-8c87-8ee558ca17bb","Type":"ContainerDied","Data":"a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6"} Jul 18 02:42:09.392785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.392506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d494d646-v9pvl" event={"ID":"e4da3129-84cd-40b9-8c87-8ee558ca17bb","Type":"ContainerDied","Data":"2096e0c713dc4263bb85796f8a5c5798e264826f78297d295ec9b6f763084ece"} Jul 18 02:42:09.392785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.392527 2560 scope.go:117] "RemoveContainer" containerID="a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6" Jul 18 02:42:09.400358 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.400342 2560 scope.go:117] "RemoveContainer" containerID="a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6" Jul 18 02:42:09.400623 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:42:09.400604 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6\": container with ID starting with a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6 not found: ID does not exist" containerID="a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6" Jul 18 02:42:09.400671 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.400630 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6"} err="failed to get container status \"a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6\": rpc error: code = NotFound desc = could not find container \"a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6\": container with ID starting with a53104dbd5299ae4a671225effe69a5f6c87e49dfd6ab6c9583dc37a673583c6 not found: ID does not exist" Jul 18 02:42:09.410834 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.410761 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d494d646-v9pvl"] Jul 18 02:42:09.413700 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:09.413679 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66d494d646-v9pvl"] Jul 18 02:42:10.755817 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:10.755785 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4da3129-84cd-40b9-8c87-8ee558ca17bb" path="/var/lib/kubelet/pods/e4da3129-84cd-40b9-8c87-8ee558ca17bb/volumes" Jul 18 02:42:37.305942 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.305905 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:42:37.306475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.306187 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="alertmanager" containerID="cri-o://155c1d4dee4a29f6b4d44634c00ef4d745583f4b2305407bbd7ebf20f717f5c5" gracePeriod=120 Jul 18 02:42:37.306475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.306272 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-metric" containerID="cri-o://8673ce649460347b058cb8012613620c84ae2c793f4eef8185799aded3455dfe" gracePeriod=120 Jul 18 02:42:37.306475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.306298 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="prom-label-proxy" containerID="cri-o://0a87fb5c9c3c8b179fb78df8d4466b2bcf92fe9c380f86ecfc688c21d5abda0c" gracePeriod=120 Jul 18 02:42:37.306475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.306318 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-web" containerID="cri-o://7602477253d80fddff171b0716f4cde381e3fdbe0474889b5a2b5fc533778a0f" gracePeriod=120 Jul 18 02:42:37.306475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.306335 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy" containerID="cri-o://4eecb1d5751271805ba22a2bf67bd3634e137785b108d534bb5b28e0678ede8c" gracePeriod=120 Jul 18 02:42:37.306475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.306407 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="config-reloader" containerID="cri-o://973e2be3cb678102853c5140ab968dbfa275a2172c703d64014a16146180e149" gracePeriod=120 Jul 18 02:42:37.495197 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495171 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="0a87fb5c9c3c8b179fb78df8d4466b2bcf92fe9c380f86ecfc688c21d5abda0c" exitCode=0 Jul 18 02:42:37.495197 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495193 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="4eecb1d5751271805ba22a2bf67bd3634e137785b108d534bb5b28e0678ede8c" exitCode=0 Jul 18 02:42:37.495197 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495200 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="973e2be3cb678102853c5140ab968dbfa275a2172c703d64014a16146180e149" exitCode=0 Jul 18 02:42:37.495354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495205 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="155c1d4dee4a29f6b4d44634c00ef4d745583f4b2305407bbd7ebf20f717f5c5" exitCode=0 Jul 18 02:42:37.495354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"0a87fb5c9c3c8b179fb78df8d4466b2bcf92fe9c380f86ecfc688c21d5abda0c"} Jul 18 02:42:37.495354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495302 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"4eecb1d5751271805ba22a2bf67bd3634e137785b108d534bb5b28e0678ede8c"} Jul 18 02:42:37.495354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495328 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"973e2be3cb678102853c5140ab968dbfa275a2172c703d64014a16146180e149"} Jul 18 02:42:37.495354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:37.495353 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"155c1d4dee4a29f6b4d44634c00ef4d745583f4b2305407bbd7ebf20f717f5c5"} Jul 18 02:42:38.503381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.503347 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="8673ce649460347b058cb8012613620c84ae2c793f4eef8185799aded3455dfe" exitCode=0 Jul 18 02:42:38.503381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.503377 2560 generic.go:356] "Generic (PLEG): container finished" podID="c77c77d8-5a86-4447-8513-170db020844e" containerID="7602477253d80fddff171b0716f4cde381e3fdbe0474889b5a2b5fc533778a0f" exitCode=0 Jul 18 02:42:38.503729 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.503413 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"8673ce649460347b058cb8012613620c84ae2c793f4eef8185799aded3455dfe"} Jul 18 02:42:38.503729 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.503481 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"7602477253d80fddff171b0716f4cde381e3fdbe0474889b5a2b5fc533778a0f"} Jul 18 02:42:38.546099 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.546079 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:38.684044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.683959 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-tls-assets\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684007 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-config-volume\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684027 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-web\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684046 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684335 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684063 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-web-config\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684335 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684080 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-cluster-tls-config\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684335 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684108 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-metrics-client-ca\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684335 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684131 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-main-tls\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684335 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684160 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-main-db\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684527 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:42:38.684590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684547 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684586 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2krz\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-kube-api-access-b2krz\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684746 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684638 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-trusted-ca-bundle\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.684746 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684680 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-config-out\") pod \"c77c77d8-5a86-4447-8513-170db020844e\" (UID: \"c77c77d8-5a86-4447-8513-170db020844e\") " Jul 18 02:42:38.685073 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.684827 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:42:38.685073 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.685028 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-metrics-client-ca\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.685073 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.685049 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-main-db\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.685314 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.685143 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:42:38.687590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.687553 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-config-out" (OuterVolumeSpecName: "config-out") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:42:38.687741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.687610 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.687741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.687648 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:42:38.687940 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.687908 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-config-volume" (OuterVolumeSpecName: "config-volume") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.688083 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.688045 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.688169 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.688132 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-kube-api-access-b2krz" (OuterVolumeSpecName: "kube-api-access-b2krz") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "kube-api-access-b2krz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:42:38.688689 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.688659 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.688835 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.688809 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.691472 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.691446 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.698890 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.698865 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-web-config" (OuterVolumeSpecName: "web-config") pod "c77c77d8-5a86-4447-8513-170db020844e" (UID: "c77c77d8-5a86-4447-8513-170db020844e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:42:38.785614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785591 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77c77d8-5a86-4447-8513-170db020844e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785611 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c77c77d8-5a86-4447-8513-170db020844e-config-out\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785621 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-tls-assets\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785628 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-config-volume\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785637 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785645 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785654 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-web-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785662 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-cluster-tls-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785670 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-main-tls\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785678 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c77c77d8-5a86-4447-8513-170db020844e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:38.785732 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:38.785687 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2krz\" (UniqueName: \"kubernetes.io/projected/c77c77d8-5a86-4447-8513-170db020844e-kube-api-access-b2krz\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:42:39.509431 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.509396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c77c77d8-5a86-4447-8513-170db020844e","Type":"ContainerDied","Data":"15d67475d73e16158acc8cbf64765bda6a08837e4385177f58885ab1164354b2"} Jul 18 02:42:39.509796 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.509450 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.509796 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.509495 2560 scope.go:117] "RemoveContainer" containerID="0a87fb5c9c3c8b179fb78df8d4466b2bcf92fe9c380f86ecfc688c21d5abda0c" Jul 18 02:42:39.516830 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.516808 2560 scope.go:117] "RemoveContainer" containerID="8673ce649460347b058cb8012613620c84ae2c793f4eef8185799aded3455dfe" Jul 18 02:42:39.523265 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.523251 2560 scope.go:117] "RemoveContainer" containerID="4eecb1d5751271805ba22a2bf67bd3634e137785b108d534bb5b28e0678ede8c" Jul 18 02:42:39.529996 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.529947 2560 scope.go:117] "RemoveContainer" containerID="7602477253d80fddff171b0716f4cde381e3fdbe0474889b5a2b5fc533778a0f" Jul 18 02:42:39.530056 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.530040 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:42:39.536063 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.536037 2560 scope.go:117] "RemoveContainer" containerID="973e2be3cb678102853c5140ab968dbfa275a2172c703d64014a16146180e149" Jul 18 02:42:39.538682 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.538666 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:42:39.542798 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.542781 2560 scope.go:117] "RemoveContainer" containerID="155c1d4dee4a29f6b4d44634c00ef4d745583f4b2305407bbd7ebf20f717f5c5" Jul 18 02:42:39.549193 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.549175 2560 scope.go:117] "RemoveContainer" containerID="bf01ab26572419a2447fded50addfaf381239b63c7db29e7bd8b7c4749f33f2c" Jul 18 02:42:39.563062 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563044 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:42:39.563280 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563268 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-web" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563280 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-web" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563292 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="config-reloader" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563298 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="config-reloader" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563305 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-metric" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563310 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-metric" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563318 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4da3129-84cd-40b9-8c87-8ee558ca17bb" containerName="console" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563323 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4da3129-84cd-40b9-8c87-8ee558ca17bb" containerName="console" Jul 18 02:42:39.563328 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563331 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36d6c74a-1633-4959-ad46-713e150cfd17" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563336 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="36d6c74a-1633-4959-ad46-713e150cfd17" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563341 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="init-config-reloader" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563346 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="init-config-reloader" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563352 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563359 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563366 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714fac21-19ce-4bb9-8fb2-afbf082bd20c" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563371 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="714fac21-19ce-4bb9-8fb2-afbf082bd20c" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563377 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="prom-label-proxy" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563382 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="prom-label-proxy" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563390 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="alertmanager" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563395 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="alertmanager" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563432 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="config-reloader" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563438 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="36d6c74a-1633-4959-ad46-713e150cfd17" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563445 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="714fac21-19ce-4bb9-8fb2-afbf082bd20c" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563450 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-web" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563456 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563462 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="alertmanager" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563467 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4da3129-84cd-40b9-8c87-8ee558ca17bb" containerName="console" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563473 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="kube-rbac-proxy-metric" Jul 18 02:42:39.563568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.563478 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c77d8-5a86-4447-8513-170db020844e" containerName="prom-label-proxy" Jul 18 02:42:39.566883 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.566868 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.571400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571087 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 18 02:42:39.571400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571185 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6qklv\"" Jul 18 02:42:39.571400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571196 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 18 02:42:39.571400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571196 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 18 02:42:39.571400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571247 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 18 02:42:39.571400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571271 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 18 02:42:39.571731 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571470 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 18 02:42:39.571731 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 18 02:42:39.571731 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.571536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 18 02:42:39.575215 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.575195 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 18 02:42:39.581887 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.581869 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:42:39.692687 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692687 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2135806a-4f69-4cb9-9f03-ffc022effd95-config-out\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692687 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2135806a-4f69-4cb9-9f03-ffc022effd95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692852 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692852 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2135806a-4f69-4cb9-9f03-ffc022effd95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692852 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-config-volume\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692852 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692770 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsblv\" (UniqueName: \"kubernetes.io/projected/2135806a-4f69-4cb9-9f03-ffc022effd95-kube-api-access-hsblv\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.692852 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692792 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.693109 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692855 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.693109 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2135806a-4f69-4cb9-9f03-ffc022effd95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.693109 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.693109 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692919 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-web-config\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.693109 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.692989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2135806a-4f69-4cb9-9f03-ffc022effd95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793583 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793561 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793638 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2135806a-4f69-4cb9-9f03-ffc022effd95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793638 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793638 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-web-config\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2135806a-4f69-4cb9-9f03-ffc022effd95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2135806a-4f69-4cb9-9f03-ffc022effd95-config-out\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2135806a-4f69-4cb9-9f03-ffc022effd95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.793788 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2135806a-4f69-4cb9-9f03-ffc022effd95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.794088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793802 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-config-volume\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.794088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsblv\" (UniqueName: \"kubernetes.io/projected/2135806a-4f69-4cb9-9f03-ffc022effd95-kube-api-access-hsblv\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.794088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.793845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.794239 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.794154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2135806a-4f69-4cb9-9f03-ffc022effd95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.795549 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.795486 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2135806a-4f69-4cb9-9f03-ffc022effd95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.796400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.796358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2135806a-4f69-4cb9-9f03-ffc022effd95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.796764 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.796722 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2135806a-4f69-4cb9-9f03-ffc022effd95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.796862 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.796812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.796925 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.796877 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-config-volume\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.796925 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.796888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.797169 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.797149 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.797243 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.797189 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-web-config\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.797298 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.797283 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.798289 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.798269 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2135806a-4f69-4cb9-9f03-ffc022effd95-config-out\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.798623 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.798605 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2135806a-4f69-4cb9-9f03-ffc022effd95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.803493 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.803476 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsblv\" (UniqueName: \"kubernetes.io/projected/2135806a-4f69-4cb9-9f03-ffc022effd95-kube-api-access-hsblv\") pod \"alertmanager-main-0\" (UID: \"2135806a-4f69-4cb9-9f03-ffc022effd95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:39.879030 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:39.879008 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 18 02:42:40.008235 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:40.008205 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 18 02:42:40.011362 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:42:40.011327 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2135806a_4f69_4cb9_9f03_ffc022effd95.slice/crio-14caa78752085482829fec25ac955fd971b9a069372633f8629a4a03ec74950d WatchSource:0}: Error finding container 14caa78752085482829fec25ac955fd971b9a069372633f8629a4a03ec74950d: Status 404 returned error can't find the container with id 14caa78752085482829fec25ac955fd971b9a069372633f8629a4a03ec74950d Jul 18 02:42:40.515443 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:40.515409 2560 generic.go:356] "Generic (PLEG): container finished" podID="2135806a-4f69-4cb9-9f03-ffc022effd95" containerID="6d3ed92761f6968f66a7b3cca665d7826d3a34a1779c8956dd1a170feb08c299" exitCode=0 Jul 18 02:42:40.515821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:40.515450 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerDied","Data":"6d3ed92761f6968f66a7b3cca665d7826d3a34a1779c8956dd1a170feb08c299"} Jul 18 02:42:40.515821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:40.515499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"14caa78752085482829fec25ac955fd971b9a069372633f8629a4a03ec74950d"} Jul 18 02:42:40.756582 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:40.756557 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c77c77d8-5a86-4447-8513-170db020844e" path="/var/lib/kubelet/pods/c77c77d8-5a86-4447-8513-170db020844e/volumes" Jul 18 02:42:41.524272 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.524228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"b48d9c4d403db4ff4730cd11be906234784791cbe9dae7f9d5b5b04341a556eb"} Jul 18 02:42:41.524651 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.524317 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"49f41a9f54c2ae97d4b48bab5558fb4431507b3db87dd6c29d3fba90196cc8cd"} Jul 18 02:42:41.524651 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.524357 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"5174ac006a0bda3b54015be667e30ef4ac97b5060573aafab599b16aef62fb65"} Jul 18 02:42:41.524651 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.524395 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"3707a3106e3f983f43f5595e01414723575be1245cafaec00700cb5418cc5d13"} Jul 18 02:42:41.524651 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.524429 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"d56ad93c9eae085f9236601eb02dd64a9df584e620995ab2188ff965d9e41b06"} Jul 18 02:42:41.524651 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.524463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2135806a-4f69-4cb9-9f03-ffc022effd95","Type":"ContainerStarted","Data":"2889610d06604a54dc32faa609378b70aad933948391957290d8acf83e1db7f8"} Jul 18 02:42:41.553963 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:41.553904 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.538977385 podStartE2EDuration="2.55389108s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.913695ms" isStatefulPod=false podCreationTimestamp="2026-07-18 02:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:42:41.552745464 +0000 UTC m=+153.341519258" watchObservedRunningTime="2026-07-18 02:42:41.55389108 +0000 UTC m=+153.342664850" Jul 18 02:42:58.755644 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.755615 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8475b8b55-5scrz"] Jul 18 02:42:58.757818 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.757803 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.766468 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.766434 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8475b8b55-5scrz"] Jul 18 02:42:58.931186 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpzwl\" (UniqueName: \"kubernetes.io/projected/3a546d2c-0853-49a5-9639-93a395bc3c53-kube-api-access-bpzwl\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.931311 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-console-config\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.931311 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-serving-cert\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.931311 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-oauth-serving-cert\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.931311 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-trusted-ca-bundle\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.931311 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931307 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-service-ca\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:58.931467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:58.931337 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-oauth-config\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032471 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-oauth-serving-cert\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032598 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-trusted-ca-bundle\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032598 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-service-ca\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032598 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-oauth-config\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032598 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpzwl\" (UniqueName: \"kubernetes.io/projected/3a546d2c-0853-49a5-9639-93a395bc3c53-kube-api-access-bpzwl\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032806 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-console-config\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.032806 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.032785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-serving-cert\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.033305 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.033278 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-oauth-serving-cert\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.033474 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.033307 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-service-ca\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.033474 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.033353 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-console-config\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.033691 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.033405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-trusted-ca-bundle\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.034940 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.034920 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-oauth-config\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.035145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.035130 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-serving-cert\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.040833 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.040809 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpzwl\" (UniqueName: \"kubernetes.io/projected/3a546d2c-0853-49a5-9639-93a395bc3c53-kube-api-access-bpzwl\") pod \"console-8475b8b55-5scrz\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.066981 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.066955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:42:59.190564 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.190535 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8475b8b55-5scrz"] Jul 18 02:42:59.192612 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:42:59.192587 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a546d2c_0853_49a5_9639_93a395bc3c53.slice/crio-53d19d2f91be72cfe52aacaeb494ebed77b5d3678da0e36b16b4e2c69005e3f2 WatchSource:0}: Error finding container 53d19d2f91be72cfe52aacaeb494ebed77b5d3678da0e36b16b4e2c69005e3f2: Status 404 returned error can't find the container with id 53d19d2f91be72cfe52aacaeb494ebed77b5d3678da0e36b16b4e2c69005e3f2 Jul 18 02:42:59.596601 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.596565 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8475b8b55-5scrz" event={"ID":"3a546d2c-0853-49a5-9639-93a395bc3c53","Type":"ContainerStarted","Data":"c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c"} Jul 18 02:42:59.596760 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.596620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8475b8b55-5scrz" event={"ID":"3a546d2c-0853-49a5-9639-93a395bc3c53","Type":"ContainerStarted","Data":"53d19d2f91be72cfe52aacaeb494ebed77b5d3678da0e36b16b4e2c69005e3f2"} Jul 18 02:42:59.614048 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:42:59.614001 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8475b8b55-5scrz" podStartSLOduration=1.6139882399999999 podStartE2EDuration="1.61398824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:42:59.613521062 +0000 UTC m=+171.402294833" watchObservedRunningTime="2026-07-18 02:42:59.61398824 +0000 UTC m=+171.402762004" Jul 18 02:43:09.067549 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:09.067519 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:43:09.068036 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:09.067578 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:43:09.072258 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:09.072235 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:43:09.638341 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:09.638310 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:43:09.686064 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:09.686030 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdd576dd4-flnt4"] Jul 18 02:43:18.137790 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.137751 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs"] Jul 18 02:43:18.142863 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.142846 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.145685 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.145666 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 02:43:18.146639 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.146623 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 02:43:18.146721 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.146658 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-768v5\"" Jul 18 02:43:18.150137 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.150118 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs"] Jul 18 02:43:18.165373 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.165344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.165470 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.165398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.165513 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.165466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llmzz\" (UniqueName: \"kubernetes.io/projected/e50efa23-746c-48c0-8f7b-b30413cf23dd-kube-api-access-llmzz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.266741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.266720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.266843 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.266770 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.266843 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.266792 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llmzz\" (UniqueName: \"kubernetes.io/projected/e50efa23-746c-48c0-8f7b-b30413cf23dd-kube-api-access-llmzz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.267100 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.267080 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.267150 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.267123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.275021 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.275001 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llmzz\" (UniqueName: \"kubernetes.io/projected/e50efa23-746c-48c0-8f7b-b30413cf23dd-kube-api-access-llmzz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.453171 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.453122 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:18.576589 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.576565 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs"] Jul 18 02:43:18.579014 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:43:18.578988 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50efa23_746c_48c0_8f7b_b30413cf23dd.slice/crio-514f4563d6e5b5ce4a00be6c2b62ca83472aec54b3ba14ca4155196a89511cfd WatchSource:0}: Error finding container 514f4563d6e5b5ce4a00be6c2b62ca83472aec54b3ba14ca4155196a89511cfd: Status 404 returned error can't find the container with id 514f4563d6e5b5ce4a00be6c2b62ca83472aec54b3ba14ca4155196a89511cfd Jul 18 02:43:18.668872 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:18.668843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" event={"ID":"e50efa23-746c-48c0-8f7b-b30413cf23dd","Type":"ContainerStarted","Data":"514f4563d6e5b5ce4a00be6c2b62ca83472aec54b3ba14ca4155196a89511cfd"} Jul 18 02:43:25.697877 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:25.697835 2560 generic.go:356] "Generic (PLEG): container finished" podID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerID="1d925e456f78f8d89b439b66459b4ef553970b785b30d24199d306eedf9b011a" exitCode=0 Jul 18 02:43:25.698270 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:25.697894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" event={"ID":"e50efa23-746c-48c0-8f7b-b30413cf23dd","Type":"ContainerDied","Data":"1d925e456f78f8d89b439b66459b4ef553970b785b30d24199d306eedf9b011a"} Jul 18 02:43:28.709979 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:28.709943 2560 generic.go:356] "Generic (PLEG): container finished" podID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerID="9cd995651f21ea2f49c6765a1b785b683dfc29cfd471a6eb03f8b70983df3f9b" exitCode=0 Jul 18 02:43:28.710397 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:28.710045 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" event={"ID":"e50efa23-746c-48c0-8f7b-b30413cf23dd","Type":"ContainerDied","Data":"9cd995651f21ea2f49c6765a1b785b683dfc29cfd471a6eb03f8b70983df3f9b"} Jul 18 02:43:34.711104 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:34.711026 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cdd576dd4-flnt4" podUID="7aa45a40-196e-4477-89a7-3328b87370bc" containerName="console" containerID="cri-o://a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a" gracePeriod=15 Jul 18 02:43:36.250511 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.250488 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:43:36.326836 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.326810 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-serving-cert\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327074 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.326867 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-trusted-ca-bundle\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327074 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.326892 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-oauth-serving-cert\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327074 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.326937 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj87z\" (UniqueName: \"kubernetes.io/projected/7aa45a40-196e-4477-89a7-3328b87370bc-kube-api-access-lj87z\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327074 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.326989 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-oauth-config\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327074 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.327050 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-service-ca\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327545 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.327098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-console-config\") pod \"7aa45a40-196e-4477-89a7-3328b87370bc\" (UID: \"7aa45a40-196e-4477-89a7-3328b87370bc\") " Jul 18 02:43:36.327545 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.327346 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:43:36.327872 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.327835 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:43:36.328082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.327896 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-service-ca" (OuterVolumeSpecName: "service-ca") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:43:36.328082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.327909 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-console-config" (OuterVolumeSpecName: "console-config") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:43:36.329258 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.329230 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:43:36.329438 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.329415 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aa45a40-196e-4477-89a7-3328b87370bc-kube-api-access-lj87z" (OuterVolumeSpecName: "kube-api-access-lj87z") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "kube-api-access-lj87z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:36.329782 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.329751 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7aa45a40-196e-4477-89a7-3328b87370bc" (UID: "7aa45a40-196e-4477-89a7-3328b87370bc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:43:36.428309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428259 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.428309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428279 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-trusted-ca-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.428309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428289 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-oauth-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.428309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428300 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj87z\" (UniqueName: \"kubernetes.io/projected/7aa45a40-196e-4477-89a7-3328b87370bc-kube-api-access-lj87z\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.428309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428309 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7aa45a40-196e-4477-89a7-3328b87370bc-console-oauth-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.428504 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428317 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-service-ca\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.428504 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.428326 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7aa45a40-196e-4477-89a7-3328b87370bc-console-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:36.744105 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.744018 2560 generic.go:356] "Generic (PLEG): container finished" podID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerID="ab9d70179c90dd405365af32c906ad825b86ec1f236c634068391a863b9ee360" exitCode=0 Jul 18 02:43:36.744243 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.744106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" event={"ID":"e50efa23-746c-48c0-8f7b-b30413cf23dd","Type":"ContainerDied","Data":"ab9d70179c90dd405365af32c906ad825b86ec1f236c634068391a863b9ee360"} Jul 18 02:43:36.745252 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.745229 2560 generic.go:356] "Generic (PLEG): container finished" podID="7aa45a40-196e-4477-89a7-3328b87370bc" containerID="a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a" exitCode=0 Jul 18 02:43:36.745360 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.745279 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd576dd4-flnt4" Jul 18 02:43:36.745360 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.745293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd576dd4-flnt4" event={"ID":"7aa45a40-196e-4477-89a7-3328b87370bc","Type":"ContainerDied","Data":"a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a"} Jul 18 02:43:36.745360 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.745332 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd576dd4-flnt4" event={"ID":"7aa45a40-196e-4477-89a7-3328b87370bc","Type":"ContainerDied","Data":"fc170cd84e79c821203341e73f9b0d05c203c2ae3cf922b2e688c6eafaa41c24"} Jul 18 02:43:36.745475 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.745358 2560 scope.go:117] "RemoveContainer" containerID="a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a" Jul 18 02:43:36.753931 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.753912 2560 scope.go:117] "RemoveContainer" containerID="a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a" Jul 18 02:43:36.754210 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:43:36.754192 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a\": container with ID starting with a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a not found: ID does not exist" containerID="a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a" Jul 18 02:43:36.754263 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.754217 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a"} err="failed to get container status \"a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a\": rpc error: code = NotFound desc = could not find container \"a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a\": container with ID starting with a263f0bee92a90c915e85199c3d8bad8efaf7065c9341916a690672fea0fbb8a not found: ID does not exist" Jul 18 02:43:36.775086 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.775061 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdd576dd4-flnt4"] Jul 18 02:43:36.778819 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:36.778801 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cdd576dd4-flnt4"] Jul 18 02:43:37.874225 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.874201 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:37.944668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.944642 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-bundle\") pod \"e50efa23-746c-48c0-8f7b-b30413cf23dd\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " Jul 18 02:43:37.944777 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.944674 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llmzz\" (UniqueName: \"kubernetes.io/projected/e50efa23-746c-48c0-8f7b-b30413cf23dd-kube-api-access-llmzz\") pod \"e50efa23-746c-48c0-8f7b-b30413cf23dd\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " Jul 18 02:43:37.944777 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.944727 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-util\") pod \"e50efa23-746c-48c0-8f7b-b30413cf23dd\" (UID: \"e50efa23-746c-48c0-8f7b-b30413cf23dd\") " Jul 18 02:43:37.945424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.945402 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-bundle" (OuterVolumeSpecName: "bundle") pod "e50efa23-746c-48c0-8f7b-b30413cf23dd" (UID: "e50efa23-746c-48c0-8f7b-b30413cf23dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:37.946769 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.946746 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50efa23-746c-48c0-8f7b-b30413cf23dd-kube-api-access-llmzz" (OuterVolumeSpecName: "kube-api-access-llmzz") pod "e50efa23-746c-48c0-8f7b-b30413cf23dd" (UID: "e50efa23-746c-48c0-8f7b-b30413cf23dd"). InnerVolumeSpecName "kube-api-access-llmzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:37.949514 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:37.949494 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-util" (OuterVolumeSpecName: "util") pod "e50efa23-746c-48c0-8f7b-b30413cf23dd" (UID: "e50efa23-746c-48c0-8f7b-b30413cf23dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:38.045395 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.045377 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:38.045489 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.045396 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e50efa23-746c-48c0-8f7b-b30413cf23dd-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:38.045489 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.045406 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llmzz\" (UniqueName: \"kubernetes.io/projected/e50efa23-746c-48c0-8f7b-b30413cf23dd-kube-api-access-llmzz\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:38.755886 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.755857 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" Jul 18 02:43:38.756095 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.755905 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aa45a40-196e-4477-89a7-3328b87370bc" path="/var/lib/kubelet/pods/7aa45a40-196e-4477-89a7-3328b87370bc/volumes" Jul 18 02:43:38.756458 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.756434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wxsrs" event={"ID":"e50efa23-746c-48c0-8f7b-b30413cf23dd","Type":"ContainerDied","Data":"514f4563d6e5b5ce4a00be6c2b62ca83472aec54b3ba14ca4155196a89511cfd"} Jul 18 02:43:38.756545 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:38.756477 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="514f4563d6e5b5ce4a00be6c2b62ca83472aec54b3ba14ca4155196a89511cfd" Jul 18 02:43:46.118357 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118319 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr"] Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118714 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="util" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118730 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="util" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118746 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="pull" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118755 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="pull" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118779 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa45a40-196e-4477-89a7-3328b87370bc" containerName="console" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118787 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa45a40-196e-4477-89a7-3328b87370bc" containerName="console" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118801 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="extract" Jul 18 02:43:46.118821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118810 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="extract" Jul 18 02:43:46.119125 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118872 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50efa23-746c-48c0-8f7b-b30413cf23dd" containerName="extract" Jul 18 02:43:46.119125 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.118889 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aa45a40-196e-4477-89a7-3328b87370bc" containerName="console" Jul 18 02:43:46.148536 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.148503 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr"] Jul 18 02:43:46.148696 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.148590 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.151142 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.151119 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 18 02:43:46.151368 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.151352 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 18 02:43:46.151456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.151438 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9chsp\"" Jul 18 02:43:46.298938 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.298908 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/056a5791-071f-43cf-a788-961980c734c4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c26zr\" (UID: \"056a5791-071f-43cf-a788-961980c734c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.299080 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.298979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9lmt\" (UniqueName: \"kubernetes.io/projected/056a5791-071f-43cf-a788-961980c734c4-kube-api-access-n9lmt\") pod \"cert-manager-operator-controller-manager-55c444bf86-c26zr\" (UID: \"056a5791-071f-43cf-a788-961980c734c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.399771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.399715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/056a5791-071f-43cf-a788-961980c734c4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c26zr\" (UID: \"056a5791-071f-43cf-a788-961980c734c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.399771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.399760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9lmt\" (UniqueName: \"kubernetes.io/projected/056a5791-071f-43cf-a788-961980c734c4-kube-api-access-n9lmt\") pod \"cert-manager-operator-controller-manager-55c444bf86-c26zr\" (UID: \"056a5791-071f-43cf-a788-961980c734c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.400163 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.400145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/056a5791-071f-43cf-a788-961980c734c4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c26zr\" (UID: \"056a5791-071f-43cf-a788-961980c734c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.407904 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.407887 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9lmt\" (UniqueName: \"kubernetes.io/projected/056a5791-071f-43cf-a788-961980c734c4-kube-api-access-n9lmt\") pod \"cert-manager-operator-controller-manager-55c444bf86-c26zr\" (UID: \"056a5791-071f-43cf-a788-961980c734c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.457395 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.457373 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" Jul 18 02:43:46.598427 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.598404 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr"] Jul 18 02:43:46.600950 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:43:46.600920 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056a5791_071f_43cf_a788_961980c734c4.slice/crio-331b1742f5a3b71583025e6eef5fed66b1668421b6e27fd2818ca5300c5b89d6 WatchSource:0}: Error finding container 331b1742f5a3b71583025e6eef5fed66b1668421b6e27fd2818ca5300c5b89d6: Status 404 returned error can't find the container with id 331b1742f5a3b71583025e6eef5fed66b1668421b6e27fd2818ca5300c5b89d6 Jul 18 02:43:46.783676 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:46.783649 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" event={"ID":"056a5791-071f-43cf-a788-961980c734c4","Type":"ContainerStarted","Data":"331b1742f5a3b71583025e6eef5fed66b1668421b6e27fd2818ca5300c5b89d6"} Jul 18 02:43:48.792255 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:48.792211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" event={"ID":"056a5791-071f-43cf-a788-961980c734c4","Type":"ContainerStarted","Data":"e930b782770cf0ee5924b23e7754de3ca3a2bc323f2edd324a3758e5ba6b78ff"} Jul 18 02:43:48.815616 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:48.815569 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c26zr" podStartSLOduration=1.43988476 podStartE2EDuration="2.815556595s" totalImagesPullingTime="1.375671835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:43:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:43:48.813120206 +0000 UTC m=+220.601893977" watchObservedRunningTime="2026-07-18 02:43:48.815556595 +0000 UTC m=+220.604330364" Jul 18 02:43:49.866778 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:49.866742 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr"] Jul 18 02:43:49.869028 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:49.869012 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:49.871389 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:49.871358 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 02:43:49.871530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:49.871405 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 02:43:49.871530 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:49.871478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-768v5\"" Jul 18 02:43:49.879911 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:49.879887 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr"] Jul 18 02:43:50.028993 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.028940 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.029156 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.029008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.029156 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.029099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk88t\" (UniqueName: \"kubernetes.io/projected/15d3933f-8250-4578-9d64-04564d831bfd-kube-api-access-hk88t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.130376 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.130315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk88t\" (UniqueName: \"kubernetes.io/projected/15d3933f-8250-4578-9d64-04564d831bfd-kube-api-access-hk88t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.130376 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.130361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.130532 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.130388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.130722 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.130702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.130782 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.130734 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.138907 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.138882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk88t\" (UniqueName: \"kubernetes.io/projected/15d3933f-8250-4578-9d64-04564d831bfd-kube-api-access-hk88t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.178674 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.178655 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:50.301659 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.301638 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr"] Jul 18 02:43:50.303504 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:43:50.303477 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d3933f_8250_4578_9d64_04564d831bfd.slice/crio-6969692d4998d6fcd722be5b321dd1eaa7f10b9f9e05c553aef2b495127539f5 WatchSource:0}: Error finding container 6969692d4998d6fcd722be5b321dd1eaa7f10b9f9e05c553aef2b495127539f5: Status 404 returned error can't find the container with id 6969692d4998d6fcd722be5b321dd1eaa7f10b9f9e05c553aef2b495127539f5 Jul 18 02:43:50.801221 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.801193 2560 generic.go:356] "Generic (PLEG): container finished" podID="15d3933f-8250-4578-9d64-04564d831bfd" containerID="a53ecc9704f2f24e3a8701d88d357ebff8c42bc41b6914bf8a3f698f2e7db8c8" exitCode=0 Jul 18 02:43:50.801380 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.801293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" event={"ID":"15d3933f-8250-4578-9d64-04564d831bfd","Type":"ContainerDied","Data":"a53ecc9704f2f24e3a8701d88d357ebff8c42bc41b6914bf8a3f698f2e7db8c8"} Jul 18 02:43:50.801380 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:50.801340 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" event={"ID":"15d3933f-8250-4578-9d64-04564d831bfd","Type":"ContainerStarted","Data":"6969692d4998d6fcd722be5b321dd1eaa7f10b9f9e05c553aef2b495127539f5"} Jul 18 02:43:53.816903 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:53.816870 2560 generic.go:356] "Generic (PLEG): container finished" podID="15d3933f-8250-4578-9d64-04564d831bfd" containerID="9aa686d87f15da0848ff90fddf34b8d36adf420b554fdbfc2c30a9da62a30145" exitCode=0 Jul 18 02:43:53.817388 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:53.816924 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" event={"ID":"15d3933f-8250-4578-9d64-04564d831bfd","Type":"ContainerDied","Data":"9aa686d87f15da0848ff90fddf34b8d36adf420b554fdbfc2c30a9da62a30145"} Jul 18 02:43:54.822134 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.822094 2560 generic.go:356] "Generic (PLEG): container finished" podID="15d3933f-8250-4578-9d64-04564d831bfd" containerID="36dea7fd770cb3a4a0e467debbc7c85846324dc5453773df95e132c5bed6b4ba" exitCode=0 Jul 18 02:43:54.822491 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.822153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" event={"ID":"15d3933f-8250-4578-9d64-04564d831bfd","Type":"ContainerDied","Data":"36dea7fd770cb3a4a0e467debbc7c85846324dc5453773df95e132c5bed6b4ba"} Jul 18 02:43:54.927790 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.927757 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj"] Jul 18 02:43:54.929987 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.929954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:54.932269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.932251 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 18 02:43:54.932497 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.932482 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 18 02:43:54.932497 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.932493 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2wc7f\"" Jul 18 02:43:54.938302 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:54.938282 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj"] Jul 18 02:43:55.071859 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.071829 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kq8cj\" (UID: \"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.071959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.071864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9ccv\" (UniqueName: \"kubernetes.io/projected/b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0-kube-api-access-x9ccv\") pod \"cert-manager-cainjector-559cdd8c87-kq8cj\" (UID: \"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.173256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.173165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kq8cj\" (UID: \"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.173256 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.173209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9ccv\" (UniqueName: \"kubernetes.io/projected/b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0-kube-api-access-x9ccv\") pod \"cert-manager-cainjector-559cdd8c87-kq8cj\" (UID: \"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.182424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.182394 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kq8cj\" (UID: \"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.182542 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.182517 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9ccv\" (UniqueName: \"kubernetes.io/projected/b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0-kube-api-access-x9ccv\") pod \"cert-manager-cainjector-559cdd8c87-kq8cj\" (UID: \"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.256309 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.256281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" Jul 18 02:43:55.388799 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.388770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj"] Jul 18 02:43:55.391949 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:43:55.391920 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb11c70cf_a7eb_42aa_99be_7bf7cddbd7b0.slice/crio-5e6965219c4b8d6af1a12fbfc49cecb557c2fbed30663bda384ac659e78f9608 WatchSource:0}: Error finding container 5e6965219c4b8d6af1a12fbfc49cecb557c2fbed30663bda384ac659e78f9608: Status 404 returned error can't find the container with id 5e6965219c4b8d6af1a12fbfc49cecb557c2fbed30663bda384ac659e78f9608 Jul 18 02:43:55.827621 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.827586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" event={"ID":"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0","Type":"ContainerStarted","Data":"5e6965219c4b8d6af1a12fbfc49cecb557c2fbed30663bda384ac659e78f9608"} Jul 18 02:43:55.939959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:55.939937 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:56.080501 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.080436 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-bundle\") pod \"15d3933f-8250-4578-9d64-04564d831bfd\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " Jul 18 02:43:56.080501 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.080483 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk88t\" (UniqueName: \"kubernetes.io/projected/15d3933f-8250-4578-9d64-04564d831bfd-kube-api-access-hk88t\") pod \"15d3933f-8250-4578-9d64-04564d831bfd\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " Jul 18 02:43:56.080650 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.080518 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-util\") pod \"15d3933f-8250-4578-9d64-04564d831bfd\" (UID: \"15d3933f-8250-4578-9d64-04564d831bfd\") " Jul 18 02:43:56.080828 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.080804 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-bundle" (OuterVolumeSpecName: "bundle") pod "15d3933f-8250-4578-9d64-04564d831bfd" (UID: "15d3933f-8250-4578-9d64-04564d831bfd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:56.082460 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.082440 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d3933f-8250-4578-9d64-04564d831bfd-kube-api-access-hk88t" (OuterVolumeSpecName: "kube-api-access-hk88t") pod "15d3933f-8250-4578-9d64-04564d831bfd" (UID: "15d3933f-8250-4578-9d64-04564d831bfd"). InnerVolumeSpecName "kube-api-access-hk88t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:43:56.084964 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.084941 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-util" (OuterVolumeSpecName: "util") pod "15d3933f-8250-4578-9d64-04564d831bfd" (UID: "15d3933f-8250-4578-9d64-04564d831bfd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:43:56.181964 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.181940 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:56.181964 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.181962 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk88t\" (UniqueName: \"kubernetes.io/projected/15d3933f-8250-4578-9d64-04564d831bfd-kube-api-access-hk88t\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:56.181964 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.181987 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d3933f-8250-4578-9d64-04564d831bfd-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:43:56.835183 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.835149 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" event={"ID":"15d3933f-8250-4578-9d64-04564d831bfd","Type":"ContainerDied","Data":"6969692d4998d6fcd722be5b321dd1eaa7f10b9f9e05c553aef2b495127539f5"} Jul 18 02:43:56.835653 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.835189 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwtrxr" Jul 18 02:43:56.835653 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.835203 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6969692d4998d6fcd722be5b321dd1eaa7f10b9f9e05c553aef2b495127539f5" Jul 18 02:43:56.843051 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843018 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6mhnx"] Jul 18 02:43:56.843307 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843291 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="util" Jul 18 02:43:56.843307 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843308 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="util" Jul 18 02:43:56.843446 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843315 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="pull" Jul 18 02:43:56.843446 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843320 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="pull" Jul 18 02:43:56.843446 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843335 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="extract" Jul 18 02:43:56.843446 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843341 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="extract" Jul 18 02:43:56.843446 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.843408 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d3933f-8250-4578-9d64-04564d831bfd" containerName="extract" Jul 18 02:43:56.847043 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.845950 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:56.848771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.848748 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4gmw8\"" Jul 18 02:43:56.853058 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.853036 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6mhnx"] Jul 18 02:43:56.989269 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.989238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04fd8433-33ad-4c55-9065-b0d2a80873d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6mhnx\" (UID: \"04fd8433-33ad-4c55-9065-b0d2a80873d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:56.989449 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:56.989292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw9l7\" (UniqueName: \"kubernetes.io/projected/04fd8433-33ad-4c55-9065-b0d2a80873d0-kube-api-access-rw9l7\") pod \"cert-manager-webhook-6486ffd997-6mhnx\" (UID: \"04fd8433-33ad-4c55-9065-b0d2a80873d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:57.090088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.089997 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04fd8433-33ad-4c55-9065-b0d2a80873d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6mhnx\" (UID: \"04fd8433-33ad-4c55-9065-b0d2a80873d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:57.090088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.090068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw9l7\" (UniqueName: \"kubernetes.io/projected/04fd8433-33ad-4c55-9065-b0d2a80873d0-kube-api-access-rw9l7\") pod \"cert-manager-webhook-6486ffd997-6mhnx\" (UID: \"04fd8433-33ad-4c55-9065-b0d2a80873d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:57.098573 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.098549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw9l7\" (UniqueName: \"kubernetes.io/projected/04fd8433-33ad-4c55-9065-b0d2a80873d0-kube-api-access-rw9l7\") pod \"cert-manager-webhook-6486ffd997-6mhnx\" (UID: \"04fd8433-33ad-4c55-9065-b0d2a80873d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:57.098684 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.098628 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04fd8433-33ad-4c55-9065-b0d2a80873d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6mhnx\" (UID: \"04fd8433-33ad-4c55-9065-b0d2a80873d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:57.159650 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.159621 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:57.302680 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.302639 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6mhnx"] Jul 18 02:43:57.304019 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:43:57.303989 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04fd8433_33ad_4c55_9065_b0d2a80873d0.slice/crio-e0ead2ff741e086939c975fd3c9ae27cb7fab62a919efdeafbb29e7f3de140e2 WatchSource:0}: Error finding container e0ead2ff741e086939c975fd3c9ae27cb7fab62a919efdeafbb29e7f3de140e2: Status 404 returned error can't find the container with id e0ead2ff741e086939c975fd3c9ae27cb7fab62a919efdeafbb29e7f3de140e2 Jul 18 02:43:57.841501 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:57.841469 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" event={"ID":"04fd8433-33ad-4c55-9065-b0d2a80873d0","Type":"ContainerStarted","Data":"e0ead2ff741e086939c975fd3c9ae27cb7fab62a919efdeafbb29e7f3de140e2"} Jul 18 02:43:58.847320 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:58.847283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" event={"ID":"b11c70cf-a7eb-42aa-99be-7bf7cddbd7b0","Type":"ContainerStarted","Data":"774b89155f6ea5f1896d94e73a76c770e18644c774ee71fa9e7b1a2bb3e2db6f"} Jul 18 02:43:58.848659 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:58.848634 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" event={"ID":"04fd8433-33ad-4c55-9065-b0d2a80873d0","Type":"ContainerStarted","Data":"4d7e5418fe06f6cfb82cefa8d11a812358edaebd2c27fedfa78cd6bd1e0169a1"} Jul 18 02:43:58.848775 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:58.848742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:43:58.865692 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:58.865397 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kq8cj" podStartSLOduration=1.936622189 podStartE2EDuration="4.865380128s" totalImagesPullingTime="2.928757939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:43:58.863503156 +0000 UTC m=+230.652276927" watchObservedRunningTime="2026-07-18 02:43:58.865380128 +0000 UTC m=+230.654153899" Jul 18 02:43:58.880187 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:43:58.880134 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" podStartSLOduration=1.863552254 podStartE2EDuration="2.880115682s" totalImagesPullingTime="1.016563428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:43:58.879664915 +0000 UTC m=+230.668438685" watchObservedRunningTime="2026-07-18 02:43:58.880115682 +0000 UTC m=+230.668889454" Jul 18 02:44:02.828664 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.828631 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj"] Jul 18 02:44:02.831767 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.831753 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:02.834215 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.834192 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 02:44:02.835042 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.835023 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 02:44:02.835139 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.835023 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fmqlz\"" Jul 18 02:44:02.841701 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.841680 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj"] Jul 18 02:44:02.935168 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.935137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0993167e-3b5d-4a6b-ae4d-a970e352487e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-x77hj\" (UID: \"0993167e-3b5d-4a6b-ae4d-a970e352487e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:02.935168 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:02.935168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2km4\" (UniqueName: \"kubernetes.io/projected/0993167e-3b5d-4a6b-ae4d-a970e352487e-kube-api-access-k2km4\") pod \"openshift-lws-operator-fd8ccff4c-x77hj\" (UID: \"0993167e-3b5d-4a6b-ae4d-a970e352487e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:03.036263 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.036233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0993167e-3b5d-4a6b-ae4d-a970e352487e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-x77hj\" (UID: \"0993167e-3b5d-4a6b-ae4d-a970e352487e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:03.036263 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.036264 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2km4\" (UniqueName: \"kubernetes.io/projected/0993167e-3b5d-4a6b-ae4d-a970e352487e-kube-api-access-k2km4\") pod \"openshift-lws-operator-fd8ccff4c-x77hj\" (UID: \"0993167e-3b5d-4a6b-ae4d-a970e352487e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:03.036606 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.036588 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0993167e-3b5d-4a6b-ae4d-a970e352487e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-x77hj\" (UID: \"0993167e-3b5d-4a6b-ae4d-a970e352487e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:03.044636 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.044606 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2km4\" (UniqueName: \"kubernetes.io/projected/0993167e-3b5d-4a6b-ae4d-a970e352487e-kube-api-access-k2km4\") pod \"openshift-lws-operator-fd8ccff4c-x77hj\" (UID: \"0993167e-3b5d-4a6b-ae4d-a970e352487e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:03.141014 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.140949 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" Jul 18 02:44:03.263905 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.263877 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj"] Jul 18 02:44:03.266228 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:03.266188 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0993167e_3b5d_4a6b_ae4d_a970e352487e.slice/crio-ba3498224dd2d6792079a9e9306dc929c5766993ff83e2b4bc88438a9c627f69 WatchSource:0}: Error finding container ba3498224dd2d6792079a9e9306dc929c5766993ff83e2b4bc88438a9c627f69: Status 404 returned error can't find the container with id ba3498224dd2d6792079a9e9306dc929c5766993ff83e2b4bc88438a9c627f69 Jul 18 02:44:03.869698 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:03.869667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" event={"ID":"0993167e-3b5d-4a6b-ae4d-a970e352487e","Type":"ContainerStarted","Data":"ba3498224dd2d6792079a9e9306dc929c5766993ff83e2b4bc88438a9c627f69"} Jul 18 02:44:04.856092 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:04.856057 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6mhnx" Jul 18 02:44:05.882332 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:05.882297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" event={"ID":"0993167e-3b5d-4a6b-ae4d-a970e352487e","Type":"ContainerStarted","Data":"1c43fd9c7f6b3806cd914980f53731bba91bcc660ee00c17236d84081e72ce49"} Jul 18 02:44:05.898652 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:05.898608 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-x77hj" podStartSLOduration=1.893656467 podStartE2EDuration="3.898593716s" totalImagesPullingTime="2.004937249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:05.897175425 +0000 UTC m=+237.685949178" watchObservedRunningTime="2026-07-18 02:44:05.898593716 +0000 UTC m=+237.687367486" Jul 18 02:44:10.680369 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.680335 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj"] Jul 18 02:44:10.683396 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.683377 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.686358 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.686330 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 02:44:10.686467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.686362 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-768v5\"" Jul 18 02:44:10.686467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.686330 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 02:44:10.693678 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.693643 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj"] Jul 18 02:44:10.798827 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.798802 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.798955 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.798857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd55s\" (UniqueName: \"kubernetes.io/projected/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-kube-api-access-pd55s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.798955 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.798922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.899282 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.899254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.899397 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.899292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.899439 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.899421 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd55s\" (UniqueName: \"kubernetes.io/projected/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-kube-api-access-pd55s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.899658 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.899641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.899693 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.899654 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.909231 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.909200 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd55s\" (UniqueName: \"kubernetes.io/projected/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-kube-api-access-pd55s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:10.992683 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:10.992608 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:11.117129 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:11.117106 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj"] Jul 18 02:44:11.119037 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:11.119007 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0c64ab_c50a_4cf7_a2b3_5c4ef7c06f6b.slice/crio-24266913f40bd97e198e30930aebae1ea108282adde1318d986d428943f24b0b WatchSource:0}: Error finding container 24266913f40bd97e198e30930aebae1ea108282adde1318d986d428943f24b0b: Status 404 returned error can't find the container with id 24266913f40bd97e198e30930aebae1ea108282adde1318d986d428943f24b0b Jul 18 02:44:11.913657 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:11.913387 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerID="28c96790fd5d6f1e5e3295f2d307010020fad4aaa9851b0da53f37cc369b7546" exitCode=0 Jul 18 02:44:11.913657 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:11.913456 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" event={"ID":"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b","Type":"ContainerDied","Data":"28c96790fd5d6f1e5e3295f2d307010020fad4aaa9851b0da53f37cc369b7546"} Jul 18 02:44:11.913657 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:11.913489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" event={"ID":"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b","Type":"ContainerStarted","Data":"24266913f40bd97e198e30930aebae1ea108282adde1318d986d428943f24b0b"} Jul 18 02:44:12.919208 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:12.919171 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerID="89df0016a1fe08f09b7a2a8831f44527bdf2efaeff6d3c08ce3ef7741e9aa3c5" exitCode=0 Jul 18 02:44:12.919655 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:12.919255 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" event={"ID":"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b","Type":"ContainerDied","Data":"89df0016a1fe08f09b7a2a8831f44527bdf2efaeff6d3c08ce3ef7741e9aa3c5"} Jul 18 02:44:13.925646 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:13.925617 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerID="6426e9fc6c7c629ec4448badcd32f54927907417a49c9ae616bf3bc37247745e" exitCode=0 Jul 18 02:44:13.925646 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:13.925650 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" event={"ID":"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b","Type":"ContainerDied","Data":"6426e9fc6c7c629ec4448badcd32f54927907417a49c9ae616bf3bc37247745e"} Jul 18 02:44:15.054747 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.054725 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:15.133009 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.132986 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd55s\" (UniqueName: \"kubernetes.io/projected/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-kube-api-access-pd55s\") pod \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " Jul 18 02:44:15.133127 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.133026 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-util\") pod \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " Jul 18 02:44:15.133127 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.133046 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-bundle\") pod \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\" (UID: \"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b\") " Jul 18 02:44:15.133765 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.133737 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-bundle" (OuterVolumeSpecName: "bundle") pod "0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" (UID: "0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:44:15.134940 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.134913 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-kube-api-access-pd55s" (OuterVolumeSpecName: "kube-api-access-pd55s") pod "0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" (UID: "0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b"). InnerVolumeSpecName "kube-api-access-pd55s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:44:15.138785 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.138765 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-util" (OuterVolumeSpecName: "util") pod "0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" (UID: "0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:44:15.233733 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.233670 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd55s\" (UniqueName: \"kubernetes.io/projected/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-kube-api-access-pd55s\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:44:15.233733 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.233692 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:44:15.233733 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.233702 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:44:15.935097 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.935073 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" Jul 18 02:44:15.935248 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.935073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmg8mj" event={"ID":"0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b","Type":"ContainerDied","Data":"24266913f40bd97e198e30930aebae1ea108282adde1318d986d428943f24b0b"} Jul 18 02:44:15.935248 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:15.935188 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24266913f40bd97e198e30930aebae1ea108282adde1318d986d428943f24b0b" Jul 18 02:44:24.012288 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012252 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz"] Jul 18 02:44:24.012772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012665 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="pull" Jul 18 02:44:24.012772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012684 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="pull" Jul 18 02:44:24.012772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012697 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="util" Jul 18 02:44:24.012772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012705 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="util" Jul 18 02:44:24.012772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012717 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="extract" Jul 18 02:44:24.012772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012724 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="extract" Jul 18 02:44:24.013097 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.012795 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0c64ab-c50a-4cf7-a2b3-5c4ef7c06f6b" containerName="extract" Jul 18 02:44:24.022923 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.022904 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.027680 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.027662 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 02:44:24.029936 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.029917 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 02:44:24.030183 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.030167 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gxrnd\"" Jul 18 02:44:24.031509 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.031497 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 02:44:24.031667 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.031650 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 02:44:24.040795 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.040775 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz"] Jul 18 02:44:24.105674 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.105648 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.105776 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.105684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.105776 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.105709 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-manifests\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.105854 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.105808 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbqlb\" (UniqueName: \"kubernetes.io/projected/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-kube-api-access-nbqlb\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.105854 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.105845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-tmp\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.206884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.206831 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbqlb\" (UniqueName: \"kubernetes.io/projected/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-kube-api-access-nbqlb\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.206884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.206872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-tmp\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.207110 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.206904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.207110 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.206937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.207110 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.206995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-manifests\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.207354 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.207337 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-manifests\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.209262 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.209239 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-tmp\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.209342 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.209324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.209412 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.209398 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.214692 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.214675 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbqlb\" (UniqueName: \"kubernetes.io/projected/4bfd634f-8e3b-4236-9659-bdd5a2ea9655-kube-api-access-nbqlb\") pod \"opendatahub-operator-controller-manager-5f9896c96c-f4klz\" (UID: \"4bfd634f-8e3b-4236-9659-bdd5a2ea9655\") " pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.333187 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.333113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:24.459443 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.459360 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz"] Jul 18 02:44:24.461724 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:24.461698 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfd634f_8e3b_4236_9659_bdd5a2ea9655.slice/crio-6afbe75e2a8fae3c2aee6edc3da19024e8f59b3920e32c0a7156e3317b8d52af WatchSource:0}: Error finding container 6afbe75e2a8fae3c2aee6edc3da19024e8f59b3920e32c0a7156e3317b8d52af: Status 404 returned error can't find the container with id 6afbe75e2a8fae3c2aee6edc3da19024e8f59b3920e32c0a7156e3317b8d52af Jul 18 02:44:24.972658 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:24.972626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" event={"ID":"4bfd634f-8e3b-4236-9659-bdd5a2ea9655","Type":"ContainerStarted","Data":"6afbe75e2a8fae3c2aee6edc3da19024e8f59b3920e32c0a7156e3317b8d52af"} Jul 18 02:44:27.988678 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:27.988643 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" event={"ID":"4bfd634f-8e3b-4236-9659-bdd5a2ea9655","Type":"ContainerStarted","Data":"02dafa9073622b9ea16624169ec0ca64aa7e8b9a75fb761c94c84f3642fca869"} Jul 18 02:44:28.994504 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:28.994412 2560 generic.go:356] "Generic (PLEG): container finished" podID="4bfd634f-8e3b-4236-9659-bdd5a2ea9655" containerID="02dafa9073622b9ea16624169ec0ca64aa7e8b9a75fb761c94c84f3642fca869" exitCode=0 Jul 18 02:44:28.994504 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:28.994478 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" event={"ID":"4bfd634f-8e3b-4236-9659-bdd5a2ea9655","Type":"ContainerDied","Data":"02dafa9073622b9ea16624169ec0ca64aa7e8b9a75fb761c94c84f3642fca869"} Jul 18 02:44:30.000086 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:30.000053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" event={"ID":"4bfd634f-8e3b-4236-9659-bdd5a2ea9655","Type":"ContainerStarted","Data":"7917a0de04b35ae6a459c3162a731c5a238e3073c6a936e7b5bf483933496542"} Jul 18 02:44:30.000492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:30.000173 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:30.024983 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:30.024925 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" podStartSLOduration=2.6900618830000003 podStartE2EDuration="7.024911763s" totalImagesPullingTime="3.264977092s" totalInitContainerRuntime="1.069872788s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:30.022858786 +0000 UTC m=+261.811632557" watchObservedRunningTime="2026-07-18 02:44:30.024911763 +0000 UTC m=+261.813685601" Jul 18 02:44:41.007778 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.007689 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f9896c96c-f4klz" Jul 18 02:44:41.459211 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.459176 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8"] Jul 18 02:44:41.463959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.463938 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.467315 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.467290 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 18 02:44:41.467428 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.467328 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f2m9s\"" Jul 18 02:44:41.467428 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.467337 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 18 02:44:41.467428 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.467346 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 18 02:44:41.474381 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.474359 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8"] Jul 18 02:44:41.542633 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.542606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/39800342-6207-473f-a2ba-06b88d388c5a-manager-config\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.542738 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.542649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl76g\" (UniqueName: \"kubernetes.io/projected/39800342-6207-473f-a2ba-06b88d388c5a-kube-api-access-fl76g\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.542738 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.542701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/39800342-6207-473f-a2ba-06b88d388c5a-metrics-cert\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.542842 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.542792 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39800342-6207-473f-a2ba-06b88d388c5a-cert\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.643842 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.643811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/39800342-6207-473f-a2ba-06b88d388c5a-manager-config\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.644004 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.643851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl76g\" (UniqueName: \"kubernetes.io/projected/39800342-6207-473f-a2ba-06b88d388c5a-kube-api-access-fl76g\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.644004 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.643880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/39800342-6207-473f-a2ba-06b88d388c5a-metrics-cert\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.644004 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.643913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39800342-6207-473f-a2ba-06b88d388c5a-cert\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.644505 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.644482 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/39800342-6207-473f-a2ba-06b88d388c5a-manager-config\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.646299 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.646278 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39800342-6207-473f-a2ba-06b88d388c5a-cert\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.646522 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.646502 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/39800342-6207-473f-a2ba-06b88d388c5a-metrics-cert\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.652932 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.652910 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl76g\" (UniqueName: \"kubernetes.io/projected/39800342-6207-473f-a2ba-06b88d388c5a-kube-api-access-fl76g\") pod \"lws-controller-manager-59945b78f9-42rj8\" (UID: \"39800342-6207-473f-a2ba-06b88d388c5a\") " pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.772743 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.772710 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:41.912348 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:41.912325 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8"] Jul 18 02:44:41.914348 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:41.914316 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39800342_6207_473f_a2ba_06b88d388c5a.slice/crio-ea375cc8ec0ad20230ba02849e2d826c389149a6684469844abf713bdd332e67 WatchSource:0}: Error finding container ea375cc8ec0ad20230ba02849e2d826c389149a6684469844abf713bdd332e67: Status 404 returned error can't find the container with id ea375cc8ec0ad20230ba02849e2d826c389149a6684469844abf713bdd332e67 Jul 18 02:44:42.053954 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:42.053854 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" event={"ID":"39800342-6207-473f-a2ba-06b88d388c5a","Type":"ContainerStarted","Data":"ea375cc8ec0ad20230ba02849e2d826c389149a6684469844abf713bdd332e67"} Jul 18 02:44:44.067898 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.067841 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" event={"ID":"39800342-6207-473f-a2ba-06b88d388c5a","Type":"ContainerStarted","Data":"9c72b10d79a8505574ec464d69578490dd638bda86d25e016881b31ada7a5527"} Jul 18 02:44:44.068410 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.067927 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:44.095342 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.095293 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" podStartSLOduration=1.5050449769999998 podStartE2EDuration="3.0952773s" totalImagesPullingTime="1.590232323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:44.094003114 +0000 UTC m=+275.882776884" watchObservedRunningTime="2026-07-18 02:44:44.0952773 +0000 UTC m=+275.884051069" Jul 18 02:44:44.931597 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.931565 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5nzlm"] Jul 18 02:44:44.934060 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.934040 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:44.937388 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.937366 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 18 02:44:44.958641 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:44.958600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5nzlm"] Jul 18 02:44:45.073204 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.073163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h7mp\" (UniqueName: \"kubernetes.io/projected/6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea-kube-api-access-6h7mp\") pod \"model-serving-api-54d89bd5fc-5nzlm\" (UID: \"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.073590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.073270 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nzlm\" (UID: \"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.174407 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.174364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nzlm\" (UID: \"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.174600 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.174578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h7mp\" (UniqueName: \"kubernetes.io/projected/6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea-kube-api-access-6h7mp\") pod \"model-serving-api-54d89bd5fc-5nzlm\" (UID: \"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.177023 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.176996 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea-tls-certs\") pod \"model-serving-api-54d89bd5fc-5nzlm\" (UID: \"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.182813 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.182732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h7mp\" (UniqueName: \"kubernetes.io/projected/6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea-kube-api-access-6h7mp\") pod \"model-serving-api-54d89bd5fc-5nzlm\" (UID: \"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea\") " pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.245346 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.245310 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:45.390801 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:45.390777 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5nzlm"] Jul 18 02:44:45.392177 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:45.392151 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6591a5a9_0b8f_45c2_9f7a_46169e6bc7ea.slice/crio-062d7cd2562cde0aed12461bbb2834a52b411c3fb86e111812e510bf00b1d5de WatchSource:0}: Error finding container 062d7cd2562cde0aed12461bbb2834a52b411c3fb86e111812e510bf00b1d5de: Status 404 returned error can't find the container with id 062d7cd2562cde0aed12461bbb2834a52b411c3fb86e111812e510bf00b1d5de Jul 18 02:44:46.080133 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:46.080089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" event={"ID":"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea","Type":"ContainerStarted","Data":"062d7cd2562cde0aed12461bbb2834a52b411c3fb86e111812e510bf00b1d5de"} Jul 18 02:44:48.090581 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:48.090541 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" event={"ID":"6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea","Type":"ContainerStarted","Data":"2349d36b7e379e6e4cff747f36676ea1af98fe5558e5370cd9bf186116b48941"} Jul 18 02:44:48.091080 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:48.090663 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:44:48.108952 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:48.108886 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" podStartSLOduration=2.411126638 podStartE2EDuration="4.108872146s" totalImagesPullingTime="1.697745508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:48.106758154 +0000 UTC m=+279.895531937" watchObservedRunningTime="2026-07-18 02:44:48.108872146 +0000 UTC m=+279.897645937" Jul 18 02:44:50.861791 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.861755 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf"] Jul 18 02:44:50.864533 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.864515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:50.867556 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.867426 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-25q9t\"" Jul 18 02:44:50.867556 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.867437 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 02:44:50.867755 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.867611 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 02:44:50.867755 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.867713 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 02:44:50.881345 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:50.881317 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf"] Jul 18 02:44:51.025574 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025539 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025741 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025720 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-data\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025898 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-token\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025898 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.025898 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.025849 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txd57\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-kube-api-access-txd57\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.034088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.034065 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg"] Jul 18 02:44:51.036247 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.036233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.055050 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.055020 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg"] Jul 18 02:44:51.127046 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.126925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-token\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127046 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.126957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127046 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.126996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txd57\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-kube-api-access-txd57\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127286 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127052 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127286 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127286 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127286 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127286 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127269 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127520 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-data\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127520 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127618 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127518 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127618 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127722 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-data\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.127775 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.127757 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.129582 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.129557 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.129723 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.129700 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.143541 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.143517 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txd57\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-kube-api-access-txd57\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.154728 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.154704 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-token\") pod \"data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.175636 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.175608 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:51.228492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsclw\" (UniqueName: \"kubernetes.io/projected/f55562f7-6c54-4e59-b9e7-0b27e71de895-kube-api-access-bsclw\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228645 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228645 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228764 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228680 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55562f7-6c54-4e59-b9e7-0b27e71de895-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228764 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228732 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228764 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228916 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228916 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.228916 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.228833 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.311191 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.311153 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf"] Jul 18 02:44:51.318689 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:51.318661 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded518773_abd8_46a8_a0b3_7221dd9c8a81.slice/crio-4bcc81d4ed968a2daf911eb681b489089f1a89516d033d8165f4f76de3f1cef4 WatchSource:0}: Error finding container 4bcc81d4ed968a2daf911eb681b489089f1a89516d033d8165f4f76de3f1cef4: Status 404 returned error can't find the container with id 4bcc81d4ed968a2daf911eb681b489089f1a89516d033d8165f4f76de3f1cef4 Jul 18 02:44:51.329322 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329285 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55562f7-6c54-4e59-b9e7-0b27e71de895-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329523 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329712 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.329878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329781 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.330424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.329930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.330424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.330002 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.330424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.330111 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.330424 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.330175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsclw\" (UniqueName: \"kubernetes.io/projected/f55562f7-6c54-4e59-b9e7-0b27e71de895-kube-api-access-bsclw\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.330808 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.330782 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55562f7-6c54-4e59-b9e7-0b27e71de895-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.332366 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.332346 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.332437 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.332386 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.345043 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.345019 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55562f7-6c54-4e59-b9e7-0b27e71de895-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.345513 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.345497 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsclw\" (UniqueName: \"kubernetes.io/projected/f55562f7-6c54-4e59-b9e7-0b27e71de895-kube-api-access-bsclw\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdswkkg\" (UID: \"f55562f7-6c54-4e59-b9e7-0b27e71de895\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.346331 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.346316 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:51.476400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:51.476368 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg"] Jul 18 02:44:51.478203 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:51.478176 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55562f7_6c54_4e59_b9e7_0b27e71de895.slice/crio-9438f895d7b46f99e390d3b771a671e112f9d612247ddbe9bfdf7033ed53b0c8 WatchSource:0}: Error finding container 9438f895d7b46f99e390d3b771a671e112f9d612247ddbe9bfdf7033ed53b0c8: Status 404 returned error can't find the container with id 9438f895d7b46f99e390d3b771a671e112f9d612247ddbe9bfdf7033ed53b0c8 Jul 18 02:44:52.111757 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:52.111716 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" event={"ID":"ed518773-abd8-46a8-a0b3-7221dd9c8a81","Type":"ContainerStarted","Data":"4bcc81d4ed968a2daf911eb681b489089f1a89516d033d8165f4f76de3f1cef4"} Jul 18 02:44:52.112910 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:52.112884 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" event={"ID":"f55562f7-6c54-4e59-b9e7-0b27e71de895","Type":"ContainerStarted","Data":"9438f895d7b46f99e390d3b771a671e112f9d612247ddbe9bfdf7033ed53b0c8"} Jul 18 02:44:53.982538 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:53.982176 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hskm5"] Jul 18 02:44:53.986661 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:53.986636 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:53.989376 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:53.989348 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 18 02:44:53.990048 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:53.989846 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-n5g55\"" Jul 18 02:44:54.004816 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.004789 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hskm5"] Jul 18 02:44:54.058184 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.058153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgfnx\" (UniqueName: \"kubernetes.io/projected/748f4be0-e25a-4abb-96b1-44599411a0a3-kube-api-access-dgfnx\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.058334 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.058285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/748f4be0-e25a-4abb-96b1-44599411a0a3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.159039 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.158993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/748f4be0-e25a-4abb-96b1-44599411a0a3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.159223 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.159082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgfnx\" (UniqueName: \"kubernetes.io/projected/748f4be0-e25a-4abb-96b1-44599411a0a3-kube-api-access-dgfnx\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.159223 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:44:54.159093 2560 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 18 02:44:54.159223 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:44:54.159169 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/748f4be0-e25a-4abb-96b1-44599411a0a3-cert podName:748f4be0-e25a-4abb-96b1-44599411a0a3 nodeName:}" failed. No retries permitted until 2026-07-18 02:44:54.65914819 +0000 UTC m=+286.447921943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/748f4be0-e25a-4abb-96b1-44599411a0a3-cert") pod "kserve-controller-manager-d6f4b65b8-hskm5" (UID: "748f4be0-e25a-4abb-96b1-44599411a0a3") : secret "kserve-webhook-server-cert" not found Jul 18 02:44:54.175882 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.175822 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgfnx\" (UniqueName: \"kubernetes.io/projected/748f4be0-e25a-4abb-96b1-44599411a0a3-kube-api-access-dgfnx\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.478009 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.477952 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:44:54.478153 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.478061 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:44:54.478153 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.478104 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:44:54.484106 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.484077 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:44:54.484207 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.484138 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:44:54.484207 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.484171 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:44:54.664166 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.664130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/748f4be0-e25a-4abb-96b1-44599411a0a3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.666726 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.666692 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/748f4be0-e25a-4abb-96b1-44599411a0a3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hskm5\" (UID: \"748f4be0-e25a-4abb-96b1-44599411a0a3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:54.900549 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:54.900515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:55.046147 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.045944 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hskm5"] Jul 18 02:44:55.046895 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:44:55.046869 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748f4be0_e25a_4abb_96b1_44599411a0a3.slice/crio-9e8807703d7c8590b7d0f82efcdae79640c934e940f0157bda0324ba9cb8a68a WatchSource:0}: Error finding container 9e8807703d7c8590b7d0f82efcdae79640c934e940f0157bda0324ba9cb8a68a: Status 404 returned error can't find the container with id 9e8807703d7c8590b7d0f82efcdae79640c934e940f0157bda0324ba9cb8a68a Jul 18 02:44:55.075450 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.075424 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59945b78f9-42rj8" Jul 18 02:44:55.145615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.134420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" event={"ID":"748f4be0-e25a-4abb-96b1-44599411a0a3","Type":"ContainerStarted","Data":"9e8807703d7c8590b7d0f82efcdae79640c934e940f0157bda0324ba9cb8a68a"} Jul 18 02:44:55.145615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.136249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" event={"ID":"ed518773-abd8-46a8-a0b3-7221dd9c8a81","Type":"ContainerStarted","Data":"d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913"} Jul 18 02:44:55.145615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.138747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" event={"ID":"f55562f7-6c54-4e59-b9e7-0b27e71de895","Type":"ContainerStarted","Data":"d1b57a75a95a5bacf03055fa1f3e019521b7c0b5eb2d6144f455812723d4e884"} Jul 18 02:44:55.172843 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.172749 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" podStartSLOduration=2.009552134 podStartE2EDuration="5.172729401s" totalImagesPullingTime="3.163177267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:55.168123768 +0000 UTC m=+286.956897539" watchObservedRunningTime="2026-07-18 02:44:55.172729401 +0000 UTC m=+286.961503174" Jul 18 02:44:55.176441 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.176410 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:44:55.177917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.177888 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 02:44:55.178063 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.177983 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 02:44:55.204202 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.204152 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" podStartSLOduration=1.206550076 podStartE2EDuration="4.204138245s" totalImagesPullingTime="2.997588169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:55.202102276 +0000 UTC m=+286.990876057" watchObservedRunningTime="2026-07-18 02:44:55.204138245 +0000 UTC m=+286.992912014" Jul 18 02:44:55.346986 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.346937 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:55.351463 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:55.351439 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:56.147243 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:56.147199 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:56.148140 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:56.148112 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdswkkg" Jul 18 02:44:56.176018 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:56.175989 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 02:44:56.176135 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:56.176040 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 02:44:56.259542 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:56.259509 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf"] Jul 18 02:44:57.176316 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:57.176269 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 18 02:44:57.176700 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:57.176331 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 18 02:44:58.158727 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:58.158692 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" event={"ID":"748f4be0-e25a-4abb-96b1-44599411a0a3","Type":"ContainerStarted","Data":"77b744e2d6d28eb082959ea05c3eb96c90319777d14f1c14a08f0fd3f16656d5"} Jul 18 02:44:58.159054 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:58.159035 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:44:58.159150 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:58.159125 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" containerID="cri-o://d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913" gracePeriod=30 Jul 18 02:44:58.174473 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:58.174429 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" podStartSLOduration=3.08439538 podStartE2EDuration="5.174415828s" totalImagesPullingTime="2.090020448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:58.173841253 +0000 UTC m=+289.962615024" watchObservedRunningTime="2026-07-18 02:44:58.174415828 +0000 UTC m=+289.963189603" Jul 18 02:44:59.098866 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:44:59.098832 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5nzlm" Jul 18 02:45:03.404681 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.404657 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:45:03.443848 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.443785 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-credential-socket\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.443848 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.443818 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-token\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.443878 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-certs\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.443927 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-podinfo\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.443998 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-envoy\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444044 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444029 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-data\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444461 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444054 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txd57\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-kube-api-access-txd57\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444461 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444073 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:45:03.444461 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444080 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-socket\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444461 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444135 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istiod-ca-cert\") pod \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\" (UID: \"ed518773-abd8-46a8-a0b3-7221dd9c8a81\") " Jul 18 02:45:03.444461 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444248 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:45:03.444461 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444459 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.444762 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444476 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.444762 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444501 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:45:03.444762 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444541 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:45:03.444762 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.444658 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:45:03.446280 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.446234 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:45:03.446393 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.446311 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:45:03.446783 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.446760 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-kube-api-access-txd57" (OuterVolumeSpecName: "kube-api-access-txd57") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "kube-api-access-txd57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:45:03.446869 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.446761 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed518773-abd8-46a8-a0b3-7221dd9c8a81" (UID: "ed518773-abd8-46a8-a0b3-7221dd9c8a81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:45:03.545426 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545395 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.545426 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545422 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.545426 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545431 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txd57\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-kube-api-access-txd57\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.545656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545440 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed518773-abd8-46a8-a0b3-7221dd9c8a81-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.545656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545449 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.545656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545457 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:03.545656 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:03.545464 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed518773-abd8-46a8-a0b3-7221dd9c8a81-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:45:04.189098 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.189062 2560 generic.go:356] "Generic (PLEG): container finished" podID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerID="d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913" exitCode=0 Jul 18 02:45:04.189279 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.189127 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" Jul 18 02:45:04.189279 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.189148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" event={"ID":"ed518773-abd8-46a8-a0b3-7221dd9c8a81","Type":"ContainerDied","Data":"d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913"} Jul 18 02:45:04.189279 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.189248 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf" event={"ID":"ed518773-abd8-46a8-a0b3-7221dd9c8a81","Type":"ContainerDied","Data":"4bcc81d4ed968a2daf911eb681b489089f1a89516d033d8165f4f76de3f1cef4"} Jul 18 02:45:04.189450 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.189279 2560 scope.go:117] "RemoveContainer" containerID="d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913" Jul 18 02:45:04.199568 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.199550 2560 scope.go:117] "RemoveContainer" containerID="d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913" Jul 18 02:45:04.199827 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:45:04.199808 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913\": container with ID starting with d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913 not found: ID does not exist" containerID="d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913" Jul 18 02:45:04.199884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.199834 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913"} err="failed to get container status \"d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913\": rpc error: code = NotFound desc = could not find container \"d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913\": container with ID starting with d361fab7b60b138535047a171b1317b4f4344ca80ae6b4b4844872ac78f9c913 not found: ID does not exist" Jul 18 02:45:04.210945 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.210919 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf"] Jul 18 02:45:04.216378 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.216359 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-f5b59fcbd-vcvvf"] Jul 18 02:45:04.757400 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:04.757353 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" path="/var/lib/kubelet/pods/ed518773-abd8-46a8-a0b3-7221dd9c8a81/volumes" Jul 18 02:45:08.624289 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:08.624260 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:45:08.624884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:08.624864 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:45:08.630779 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:08.630763 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 02:45:29.170501 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:45:29.170471 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hskm5" Jul 18 02:46:00.165082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.165049 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz"] Jul 18 02:46:00.165609 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.165528 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" Jul 18 02:46:00.165609 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.165547 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" Jul 18 02:46:00.165735 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.165641 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed518773-abd8-46a8-a0b3-7221dd9c8a81" containerName="istio-proxy" Jul 18 02:46:00.168837 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.168818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.171771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.171750 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 02:46:00.171867 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.171776 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wr5wb\"" Jul 18 02:46:00.172738 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.172724 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 02:46:00.179082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.179060 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz"] Jul 18 02:46:00.181083 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.181061 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.181188 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.181127 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.181188 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.181158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbb7d\" (UniqueName: \"kubernetes.io/projected/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-kube-api-access-cbb7d\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.282179 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.282154 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.282297 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.282186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbb7d\" (UniqueName: \"kubernetes.io/projected/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-kube-api-access-cbb7d\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.282297 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.282239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.282498 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.282481 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.282537 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.282517 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.291513 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.291494 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbb7d\" (UniqueName: \"kubernetes.io/projected/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-kube-api-access-cbb7d\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.479211 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.479114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:00.575053 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.575021 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq"] Jul 18 02:46:00.579280 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.579256 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.595395 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.595371 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq"] Jul 18 02:46:00.619394 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.619370 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz"] Jul 18 02:46:00.621009 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:00.620957 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18832bc_eed9_41f8_974f_5ae7d02aa3b2.slice/crio-1abdd00011469d5f245e2b33d1f6103e41e11eaa6ea3d7c63e2d4085a52656df WatchSource:0}: Error finding container 1abdd00011469d5f245e2b33d1f6103e41e11eaa6ea3d7c63e2d4085a52656df: Status 404 returned error can't find the container with id 1abdd00011469d5f245e2b33d1f6103e41e11eaa6ea3d7c63e2d4085a52656df Jul 18 02:46:00.684684 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.684658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4w4z\" (UniqueName: \"kubernetes.io/projected/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-kube-api-access-w4w4z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.684773 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.684690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.684825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.684796 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.786003 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.785952 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4w4z\" (UniqueName: \"kubernetes.io/projected/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-kube-api-access-w4w4z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.786145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.786012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.786145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.786137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.786315 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.786295 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.786470 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.786451 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.796497 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.796475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4w4z\" (UniqueName: \"kubernetes.io/projected/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-kube-api-access-w4w4z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.891404 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.891381 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:00.959200 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.959175 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl"] Jul 18 02:46:00.962880 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.962860 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:00.972665 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.972603 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl"] Jul 18 02:46:00.988296 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.988274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:00.988418 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.988302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:00.988418 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:00.988325 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcvqx\" (UniqueName: \"kubernetes.io/projected/c9ce62a1-c705-4621-a97e-85775e90df52-kube-api-access-vcvqx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.023028 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.023004 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq"] Jul 18 02:46:01.024906 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:01.024883 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4213b8_47d0_473a_a9b8_19a4f91fcc3b.slice/crio-e709d76b294c24d0eb6a106c35d837b6b1dabf8de8485d87e8d81b778c6f70a4 WatchSource:0}: Error finding container e709d76b294c24d0eb6a106c35d837b6b1dabf8de8485d87e8d81b778c6f70a4: Status 404 returned error can't find the container with id e709d76b294c24d0eb6a106c35d837b6b1dabf8de8485d87e8d81b778c6f70a4 Jul 18 02:46:01.089550 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.089522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.089641 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.089559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.089641 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.089582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcvqx\" (UniqueName: \"kubernetes.io/projected/c9ce62a1-c705-4621-a97e-85775e90df52-kube-api-access-vcvqx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.089884 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.089866 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.089926 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.089913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.099569 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.099546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcvqx\" (UniqueName: \"kubernetes.io/projected/c9ce62a1-c705-4621-a97e-85775e90df52-kube-api-access-vcvqx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.274594 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.274570 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:01.401621 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.401563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl"] Jul 18 02:46:01.404140 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:01.404109 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ce62a1_c705_4621_a97e_85775e90df52.slice/crio-091b932972cafd63199c6a750e11e20c3b78150b00f251ae5aa9c1116124545c WatchSource:0}: Error finding container 091b932972cafd63199c6a750e11e20c3b78150b00f251ae5aa9c1116124545c: Status 404 returned error can't find the container with id 091b932972cafd63199c6a750e11e20c3b78150b00f251ae5aa9c1116124545c Jul 18 02:46:01.462858 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.462826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" event={"ID":"c9ce62a1-c705-4621-a97e-85775e90df52","Type":"ContainerStarted","Data":"091b932972cafd63199c6a750e11e20c3b78150b00f251ae5aa9c1116124545c"} Jul 18 02:46:01.464479 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.464455 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerID="e4b8e148b796be69935f2b52a53729d631ee03499ca96b4c990abcecc023bd0f" exitCode=0 Jul 18 02:46:01.464584 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.464524 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" event={"ID":"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b","Type":"ContainerDied","Data":"e4b8e148b796be69935f2b52a53729d631ee03499ca96b4c990abcecc023bd0f"} Jul 18 02:46:01.464584 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.464563 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" event={"ID":"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b","Type":"ContainerStarted","Data":"e709d76b294c24d0eb6a106c35d837b6b1dabf8de8485d87e8d81b778c6f70a4"} Jul 18 02:46:01.465113 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.465093 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:46:01.466192 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.466167 2560 generic.go:356] "Generic (PLEG): container finished" podID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerID="7dd454ed150c9a1ea780e4f2963201386b466d9e2b95cb93802bac33eb3450e0" exitCode=0 Jul 18 02:46:01.466308 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.466211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" event={"ID":"b18832bc-eed9-41f8-974f-5ae7d02aa3b2","Type":"ContainerDied","Data":"7dd454ed150c9a1ea780e4f2963201386b466d9e2b95cb93802bac33eb3450e0"} Jul 18 02:46:01.466308 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.466244 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" event={"ID":"b18832bc-eed9-41f8-974f-5ae7d02aa3b2","Type":"ContainerStarted","Data":"1abdd00011469d5f245e2b33d1f6103e41e11eaa6ea3d7c63e2d4085a52656df"} Jul 18 02:46:01.555537 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.555483 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g"] Jul 18 02:46:01.558909 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.558892 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.571951 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.571926 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g"] Jul 18 02:46:01.594498 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.594474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.594600 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.594585 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.594652 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.594613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4dqg\" (UniqueName: \"kubernetes.io/projected/dd1266ca-1d8a-45f3-92e8-25050d26301e-kube-api-access-v4dqg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.695092 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.695070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.695199 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.695164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.695199 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.695196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4dqg\" (UniqueName: \"kubernetes.io/projected/dd1266ca-1d8a-45f3-92e8-25050d26301e-kube-api-access-v4dqg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.695425 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.695409 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.695458 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.695441 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.705382 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.705356 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4dqg\" (UniqueName: \"kubernetes.io/projected/dd1266ca-1d8a-45f3-92e8-25050d26301e-kube-api-access-v4dqg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:01.878567 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:01.878506 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:02.008478 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:02.008454 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g"] Jul 18 02:46:02.009651 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:02.009626 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1266ca_1d8a_45f3_92e8_25050d26301e.slice/crio-fe55da38222fa993304401f412235455ff02f458928e8d83b226a07b8032a982 WatchSource:0}: Error finding container fe55da38222fa993304401f412235455ff02f458928e8d83b226a07b8032a982: Status 404 returned error can't find the container with id fe55da38222fa993304401f412235455ff02f458928e8d83b226a07b8032a982 Jul 18 02:46:02.474005 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:02.473742 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerID="fa306eecea6c7d43a468da79e35bbd60480b48536f6daefccdab27e264328965" exitCode=0 Jul 18 02:46:02.474379 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:02.474116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" event={"ID":"dd1266ca-1d8a-45f3-92e8-25050d26301e","Type":"ContainerDied","Data":"fa306eecea6c7d43a468da79e35bbd60480b48536f6daefccdab27e264328965"} Jul 18 02:46:02.474379 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:02.474168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" event={"ID":"dd1266ca-1d8a-45f3-92e8-25050d26301e","Type":"ContainerStarted","Data":"fe55da38222fa993304401f412235455ff02f458928e8d83b226a07b8032a982"} Jul 18 02:46:02.475891 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:02.475867 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9ce62a1-c705-4621-a97e-85775e90df52" containerID="9cc66b772a0aaa7e66c219614f4ae2ddbe337e01712866c1cb9acd9eb859b88f" exitCode=0 Jul 18 02:46:02.476005 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:02.475955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" event={"ID":"c9ce62a1-c705-4621-a97e-85775e90df52","Type":"ContainerDied","Data":"9cc66b772a0aaa7e66c219614f4ae2ddbe337e01712866c1cb9acd9eb859b88f"} Jul 18 02:46:03.482482 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.482416 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerID="71be0b395e7c78175bc17a254ea551d268d651d41cc2f28c19adb19f2e3c3c75" exitCode=0 Jul 18 02:46:03.482812 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.482502 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" event={"ID":"dd1266ca-1d8a-45f3-92e8-25050d26301e","Type":"ContainerDied","Data":"71be0b395e7c78175bc17a254ea551d268d651d41cc2f28c19adb19f2e3c3c75"} Jul 18 02:46:03.484478 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.484453 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9ce62a1-c705-4621-a97e-85775e90df52" containerID="d97dd8d3b65727d5794e6860a56216043164eeb48eab271a85100d4b7709d878" exitCode=0 Jul 18 02:46:03.484584 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.484530 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" event={"ID":"c9ce62a1-c705-4621-a97e-85775e90df52","Type":"ContainerDied","Data":"d97dd8d3b65727d5794e6860a56216043164eeb48eab271a85100d4b7709d878"} Jul 18 02:46:03.486388 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.486368 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerID="501e63b29992d0d62d7ec441eee5527a54e54e6be51f3bc7082839ba48cde3e6" exitCode=0 Jul 18 02:46:03.486469 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.486444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" event={"ID":"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b","Type":"ContainerDied","Data":"501e63b29992d0d62d7ec441eee5527a54e54e6be51f3bc7082839ba48cde3e6"} Jul 18 02:46:03.488435 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.488417 2560 generic.go:356] "Generic (PLEG): container finished" podID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerID="f8499ec7e3dc720ec42c4a0db4b74a564b9b55b435d79fe69769bd0dd835c8ed" exitCode=0 Jul 18 02:46:03.488539 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:03.488452 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" event={"ID":"b18832bc-eed9-41f8-974f-5ae7d02aa3b2","Type":"ContainerDied","Data":"f8499ec7e3dc720ec42c4a0db4b74a564b9b55b435d79fe69769bd0dd835c8ed"} Jul 18 02:46:04.496368 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.496335 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerID="076db8e7acb9d42e706e0e0285753de9e908859fdfaf8c44e07a7a6c589f9767" exitCode=0 Jul 18 02:46:04.496769 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.496431 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" event={"ID":"dd1266ca-1d8a-45f3-92e8-25050d26301e","Type":"ContainerDied","Data":"076db8e7acb9d42e706e0e0285753de9e908859fdfaf8c44e07a7a6c589f9767"} Jul 18 02:46:04.498559 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.498536 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9ce62a1-c705-4621-a97e-85775e90df52" containerID="17f48a915f047be201db2ac33c7febaf961836b45ae5148ac765e78ac8087655" exitCode=0 Jul 18 02:46:04.498686 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.498581 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" event={"ID":"c9ce62a1-c705-4621-a97e-85775e90df52","Type":"ContainerDied","Data":"17f48a915f047be201db2ac33c7febaf961836b45ae5148ac765e78ac8087655"} Jul 18 02:46:04.500721 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.500698 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerID="4bc3a876bffe220f9f9534499e056cc757c39ccfc60fd88fa577d6eb93d212a8" exitCode=0 Jul 18 02:46:04.500806 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.500773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" event={"ID":"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b","Type":"ContainerDied","Data":"4bc3a876bffe220f9f9534499e056cc757c39ccfc60fd88fa577d6eb93d212a8"} Jul 18 02:46:04.502736 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.502710 2560 generic.go:356] "Generic (PLEG): container finished" podID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerID="bb2ca64800a3c68cbac9c249274d8a29922f878f701bfd03555b35afdecb2ab6" exitCode=0 Jul 18 02:46:04.502825 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:04.502739 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" event={"ID":"b18832bc-eed9-41f8-974f-5ae7d02aa3b2","Type":"ContainerDied","Data":"bb2ca64800a3c68cbac9c249274d8a29922f878f701bfd03555b35afdecb2ab6"} Jul 18 02:46:05.645865 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.645837 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:05.686167 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.686139 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:05.711590 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.711563 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:05.716564 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.716539 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:05.731299 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731272 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcvqx\" (UniqueName: \"kubernetes.io/projected/c9ce62a1-c705-4621-a97e-85775e90df52-kube-api-access-vcvqx\") pod \"c9ce62a1-c705-4621-a97e-85775e90df52\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " Jul 18 02:46:05.731398 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731310 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-util\") pod \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " Jul 18 02:46:05.731398 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731330 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-util\") pod \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " Jul 18 02:46:05.731398 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731355 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-bundle\") pod \"c9ce62a1-c705-4621-a97e-85775e90df52\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " Jul 18 02:46:05.731559 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731505 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-util\") pod \"c9ce62a1-c705-4621-a97e-85775e90df52\" (UID: \"c9ce62a1-c705-4621-a97e-85775e90df52\") " Jul 18 02:46:05.731615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731565 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-bundle\") pod \"dd1266ca-1d8a-45f3-92e8-25050d26301e\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " Jul 18 02:46:05.731615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731604 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-bundle\") pod \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " Jul 18 02:46:05.731717 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731665 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-bundle\") pod \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " Jul 18 02:46:05.731770 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731723 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbb7d\" (UniqueName: \"kubernetes.io/projected/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-kube-api-access-cbb7d\") pod \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\" (UID: \"b18832bc-eed9-41f8-974f-5ae7d02aa3b2\") " Jul 18 02:46:05.731770 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731756 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4w4z\" (UniqueName: \"kubernetes.io/projected/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-kube-api-access-w4w4z\") pod \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\" (UID: \"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b\") " Jul 18 02:46:05.731869 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731782 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-util\") pod \"dd1266ca-1d8a-45f3-92e8-25050d26301e\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " Jul 18 02:46:05.731869 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731810 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4dqg\" (UniqueName: \"kubernetes.io/projected/dd1266ca-1d8a-45f3-92e8-25050d26301e-kube-api-access-v4dqg\") pod \"dd1266ca-1d8a-45f3-92e8-25050d26301e\" (UID: \"dd1266ca-1d8a-45f3-92e8-25050d26301e\") " Jul 18 02:46:05.732007 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.731956 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-bundle" (OuterVolumeSpecName: "bundle") pod "c9ce62a1-c705-4621-a97e-85775e90df52" (UID: "c9ce62a1-c705-4621-a97e-85775e90df52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.732230 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.732211 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.734525 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.732441 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-bundle" (OuterVolumeSpecName: "bundle") pod "b18832bc-eed9-41f8-974f-5ae7d02aa3b2" (UID: "b18832bc-eed9-41f8-974f-5ae7d02aa3b2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.734525 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.732577 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-bundle" (OuterVolumeSpecName: "bundle") pod "dd1266ca-1d8a-45f3-92e8-25050d26301e" (UID: "dd1266ca-1d8a-45f3-92e8-25050d26301e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.734525 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.732627 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-bundle" (OuterVolumeSpecName: "bundle") pod "ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" (UID: "ab4213b8-47d0-473a-a9b8-19a4f91fcc3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.736155 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.735587 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ce62a1-c705-4621-a97e-85775e90df52-kube-api-access-vcvqx" (OuterVolumeSpecName: "kube-api-access-vcvqx") pod "c9ce62a1-c705-4621-a97e-85775e90df52" (UID: "c9ce62a1-c705-4621-a97e-85775e90df52"). InnerVolumeSpecName "kube-api-access-vcvqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:46:05.736155 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.735944 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd1266ca-1d8a-45f3-92e8-25050d26301e-kube-api-access-v4dqg" (OuterVolumeSpecName: "kube-api-access-v4dqg") pod "dd1266ca-1d8a-45f3-92e8-25050d26301e" (UID: "dd1266ca-1d8a-45f3-92e8-25050d26301e"). InnerVolumeSpecName "kube-api-access-v4dqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:46:05.736155 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.736086 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-kube-api-access-w4w4z" (OuterVolumeSpecName: "kube-api-access-w4w4z") pod "ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" (UID: "ab4213b8-47d0-473a-a9b8-19a4f91fcc3b"). InnerVolumeSpecName "kube-api-access-w4w4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:46:05.736155 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.736122 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-kube-api-access-cbb7d" (OuterVolumeSpecName: "kube-api-access-cbb7d") pod "b18832bc-eed9-41f8-974f-5ae7d02aa3b2" (UID: "b18832bc-eed9-41f8-974f-5ae7d02aa3b2"). InnerVolumeSpecName "kube-api-access-cbb7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:46:05.739018 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.738879 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-util" (OuterVolumeSpecName: "util") pod "b18832bc-eed9-41f8-974f-5ae7d02aa3b2" (UID: "b18832bc-eed9-41f8-974f-5ae7d02aa3b2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.740095 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.740073 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-util" (OuterVolumeSpecName: "util") pod "c9ce62a1-c705-4621-a97e-85775e90df52" (UID: "c9ce62a1-c705-4621-a97e-85775e90df52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.741109 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.741073 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-util" (OuterVolumeSpecName: "util") pod "ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" (UID: "ab4213b8-47d0-473a-a9b8-19a4f91fcc3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.741644 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.741629 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-util" (OuterVolumeSpecName: "util") pod "dd1266ca-1d8a-45f3-92e8-25050d26301e" (UID: "dd1266ca-1d8a-45f3-92e8-25050d26301e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:46:05.833257 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833226 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4w4z\" (UniqueName: \"kubernetes.io/projected/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-kube-api-access-w4w4z\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833257 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833258 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833290 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4dqg\" (UniqueName: \"kubernetes.io/projected/dd1266ca-1d8a-45f3-92e8-25050d26301e-kube-api-access-v4dqg\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833306 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcvqx\" (UniqueName: \"kubernetes.io/projected/c9ce62a1-c705-4621-a97e-85775e90df52-kube-api-access-vcvqx\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833319 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833327 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833347 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ce62a1-c705-4621-a97e-85775e90df52-util\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833359 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd1266ca-1d8a-45f3-92e8-25050d26301e-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833371 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab4213b8-47d0-473a-a9b8-19a4f91fcc3b-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833385 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:05.833492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:05.833398 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbb7d\" (UniqueName: \"kubernetes.io/projected/b18832bc-eed9-41f8-974f-5ae7d02aa3b2-kube-api-access-cbb7d\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:46:06.519991 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.519953 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" Jul 18 02:46:06.520170 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.519987 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl" event={"ID":"c9ce62a1-c705-4621-a97e-85775e90df52","Type":"ContainerDied","Data":"091b932972cafd63199c6a750e11e20c3b78150b00f251ae5aa9c1116124545c"} Jul 18 02:46:06.520170 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.520061 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="091b932972cafd63199c6a750e11e20c3b78150b00f251ae5aa9c1116124545c" Jul 18 02:46:06.521901 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.521878 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" event={"ID":"ab4213b8-47d0-473a-a9b8-19a4f91fcc3b","Type":"ContainerDied","Data":"e709d76b294c24d0eb6a106c35d837b6b1dabf8de8485d87e8d81b778c6f70a4"} Jul 18 02:46:06.521901 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.521896 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq" Jul 18 02:46:06.522093 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.521913 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e709d76b294c24d0eb6a106c35d837b6b1dabf8de8485d87e8d81b778c6f70a4" Jul 18 02:46:06.523726 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.523700 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" Jul 18 02:46:06.523726 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.523705 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz" event={"ID":"b18832bc-eed9-41f8-974f-5ae7d02aa3b2","Type":"ContainerDied","Data":"1abdd00011469d5f245e2b33d1f6103e41e11eaa6ea3d7c63e2d4085a52656df"} Jul 18 02:46:06.523874 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.523752 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1abdd00011469d5f245e2b33d1f6103e41e11eaa6ea3d7c63e2d4085a52656df" Jul 18 02:46:06.525638 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.525616 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" Jul 18 02:46:06.525739 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.525636 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g" event={"ID":"dd1266ca-1d8a-45f3-92e8-25050d26301e","Type":"ContainerDied","Data":"fe55da38222fa993304401f412235455ff02f458928e8d83b226a07b8032a982"} Jul 18 02:46:06.525739 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:06.525668 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe55da38222fa993304401f412235455ff02f458928e8d83b226a07b8032a982" Jul 18 02:46:19.901082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901048 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c"] Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901546 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="pull" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901561 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="pull" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901570 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="util" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901575 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="util" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901586 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="extract" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901591 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="extract" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901599 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="util" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901604 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="util" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901614 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="pull" Jul 18 02:46:19.901615 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901619 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="pull" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901626 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="util" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901631 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="util" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901642 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="util" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901648 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="util" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901654 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="pull" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901658 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="pull" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901664 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901669 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901674 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901679 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901685 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="pull" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901689 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="pull" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901695 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901699 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901756 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18832bc-eed9-41f8-974f-5ae7d02aa3b2" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901763 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ce62a1-c705-4621-a97e-85775e90df52" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901772 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd1266ca-1d8a-45f3-92e8-25050d26301e" containerName="extract" Jul 18 02:46:19.901961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.901780 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab4213b8-47d0-473a-a9b8-19a4f91fcc3b" containerName="extract" Jul 18 02:46:19.905556 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.905540 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:19.908329 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.908301 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 02:46:19.908329 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.908305 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 02:46:19.909191 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.909165 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ckvl8\"" Jul 18 02:46:19.914439 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.914419 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c"] Jul 18 02:46:19.938668 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:19.938638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfm28\" (UniqueName: \"kubernetes.io/projected/121c571f-0211-4e7c-8cca-5a7a4064ab06-kube-api-access-bfm28\") pod \"authorino-operator-75cc8d5cb7-lxd8c\" (UID: \"121c571f-0211-4e7c-8cca-5a7a4064ab06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:20.039307 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:20.039279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfm28\" (UniqueName: \"kubernetes.io/projected/121c571f-0211-4e7c-8cca-5a7a4064ab06-kube-api-access-bfm28\") pod \"authorino-operator-75cc8d5cb7-lxd8c\" (UID: \"121c571f-0211-4e7c-8cca-5a7a4064ab06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:20.050933 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:20.050914 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfm28\" (UniqueName: \"kubernetes.io/projected/121c571f-0211-4e7c-8cca-5a7a4064ab06-kube-api-access-bfm28\") pod \"authorino-operator-75cc8d5cb7-lxd8c\" (UID: \"121c571f-0211-4e7c-8cca-5a7a4064ab06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:20.216337 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:20.216272 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:20.342486 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:20.342460 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c"] Jul 18 02:46:20.344460 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:20.344433 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121c571f_0211_4e7c_8cca_5a7a4064ab06.slice/crio-842a115b9480bcd2f1e5c87bdeeea623aac63a8012887667c25b985fdc51da2e WatchSource:0}: Error finding container 842a115b9480bcd2f1e5c87bdeeea623aac63a8012887667c25b985fdc51da2e: Status 404 returned error can't find the container with id 842a115b9480bcd2f1e5c87bdeeea623aac63a8012887667c25b985fdc51da2e Jul 18 02:46:20.594264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:20.594234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" event={"ID":"121c571f-0211-4e7c-8cca-5a7a4064ab06","Type":"ContainerStarted","Data":"842a115b9480bcd2f1e5c87bdeeea623aac63a8012887667c25b985fdc51da2e"} Jul 18 02:46:23.611897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:23.611856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" event={"ID":"121c571f-0211-4e7c-8cca-5a7a4064ab06","Type":"ContainerStarted","Data":"f0f28ea506aabdad4c309841e7231ed0cf4e3ce7647c31df5a916c496043c0df"} Jul 18 02:46:23.612331 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:23.612065 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:23.634856 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:23.634807 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" podStartSLOduration=2.390613767 podStartE2EDuration="4.634793437s" totalImagesPullingTime="2.24417967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:46:23.632478493 +0000 UTC m=+375.421252263" watchObservedRunningTime="2026-07-18 02:46:23.634793437 +0000 UTC m=+375.423567207" Jul 18 02:46:24.734390 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.734344 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q"] Jul 18 02:46:24.738303 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.738284 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.741048 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.741017 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 18 02:46:24.741168 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.741075 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wr5wb\"" Jul 18 02:46:24.741168 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.741088 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 18 02:46:24.749823 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.749801 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q"] Jul 18 02:46:24.780841 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.780819 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b69b593-57b4-47d5-bc36-426f20c8570f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.780991 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.780862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b69b593-57b4-47d5-bc36-426f20c8570f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.781056 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.780991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzljd\" (UniqueName: \"kubernetes.io/projected/7b69b593-57b4-47d5-bc36-426f20c8570f-kube-api-access-gzljd\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.881725 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.881696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzljd\" (UniqueName: \"kubernetes.io/projected/7b69b593-57b4-47d5-bc36-426f20c8570f-kube-api-access-gzljd\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.881842 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.881755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b69b593-57b4-47d5-bc36-426f20c8570f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.881842 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.881780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b69b593-57b4-47d5-bc36-426f20c8570f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.881919 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:46:24.881867 2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 18 02:46:24.881952 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:46:24.881940 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b69b593-57b4-47d5-bc36-426f20c8570f-plugin-serving-cert podName:7b69b593-57b4-47d5-bc36-426f20c8570f nodeName:}" failed. No retries permitted until 2026-07-18 02:46:25.381920286 +0000 UTC m=+377.170694041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7b69b593-57b4-47d5-bc36-426f20c8570f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-dng5q" (UID: "7b69b593-57b4-47d5-bc36-426f20c8570f") : secret "plugin-serving-cert" not found Jul 18 02:46:24.882441 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.882425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b69b593-57b4-47d5-bc36-426f20c8570f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:24.892086 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:24.892059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzljd\" (UniqueName: \"kubernetes.io/projected/7b69b593-57b4-47d5-bc36-426f20c8570f-kube-api-access-gzljd\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:25.385893 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:25.385862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b69b593-57b4-47d5-bc36-426f20c8570f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:25.388128 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:25.388105 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b69b593-57b4-47d5-bc36-426f20c8570f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dng5q\" (UID: \"7b69b593-57b4-47d5-bc36-426f20c8570f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:25.649916 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:25.649857 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" Jul 18 02:46:25.781696 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:25.781673 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q"] Jul 18 02:46:25.783668 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:25.783626 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b69b593_57b4_47d5_bc36_426f20c8570f.slice/crio-3213de37df98a7dd479942b72cc3a0069f19517f41d92443e43cb71c0ecc6a8d WatchSource:0}: Error finding container 3213de37df98a7dd479942b72cc3a0069f19517f41d92443e43cb71c0ecc6a8d: Status 404 returned error can't find the container with id 3213de37df98a7dd479942b72cc3a0069f19517f41d92443e43cb71c0ecc6a8d Jul 18 02:46:26.629294 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:26.629260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" event={"ID":"7b69b593-57b4-47d5-bc36-426f20c8570f","Type":"ContainerStarted","Data":"3213de37df98a7dd479942b72cc3a0069f19517f41d92443e43cb71c0ecc6a8d"} Jul 18 02:46:34.619376 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:34.619334 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lxd8c" Jul 18 02:46:41.975192 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:41.975158 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dc4b576ff-gmjwn"] Jul 18 02:46:42.061739 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.061705 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc4b576ff-gmjwn"] Jul 18 02:46:42.061909 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.061797 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153270 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153234 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-service-ca\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153311 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-config\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153456 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-oauth-config\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153555 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-oauth-serving-cert\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153555 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-kube-api-access-bcl6m\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153645 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-trusted-ca-bundle\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.153689 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.153671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-serving-cert\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254744 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-config\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254744 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-oauth-config\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254767 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-oauth-serving-cert\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254787 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-kube-api-access-bcl6m\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254814 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-trusted-ca-bundle\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-serving-cert\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.254959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.254871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-service-ca\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.255567 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.255542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-config\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.255726 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.255703 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-oauth-serving-cert\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.255815 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.255793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-service-ca\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.255950 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.255925 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-trusted-ca-bundle\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.257592 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.257568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-oauth-config\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.257738 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.257715 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-console-serving-cert\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.270903 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.270880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/853b346b-9b40-42d6-9fbf-aa2c98c28fd7-kube-api-access-bcl6m\") pod \"console-6dc4b576ff-gmjwn\" (UID: \"853b346b-9b40-42d6-9fbf-aa2c98c28fd7\") " pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:42.373197 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:42.373148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:46.451646 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:46.451623 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc4b576ff-gmjwn"] Jul 18 02:46:46.454299 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:46:46.454275 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853b346b_9b40_42d6_9fbf_aa2c98c28fd7.slice/crio-28c282fa573a68301dc96c5a0e20abd3cb7a7712f3425ecb794c23310a468554 WatchSource:0}: Error finding container 28c282fa573a68301dc96c5a0e20abd3cb7a7712f3425ecb794c23310a468554: Status 404 returned error can't find the container with id 28c282fa573a68301dc96c5a0e20abd3cb7a7712f3425ecb794c23310a468554 Jul 18 02:46:46.745363 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:46.745327 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" event={"ID":"7b69b593-57b4-47d5-bc36-426f20c8570f","Type":"ContainerStarted","Data":"e9bb4ad235eb6e46d0fc542b88a96762990de38f1d905d73809605deb07ec046"} Jul 18 02:46:46.746929 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:46.746905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc4b576ff-gmjwn" event={"ID":"853b346b-9b40-42d6-9fbf-aa2c98c28fd7","Type":"ContainerStarted","Data":"8ca75dfef496ae5bc0582b97eb22c39cf77b70dde586cb68c4439f185be6c4fe"} Jul 18 02:46:46.747030 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:46.746947 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc4b576ff-gmjwn" event={"ID":"853b346b-9b40-42d6-9fbf-aa2c98c28fd7","Type":"ContainerStarted","Data":"28c282fa573a68301dc96c5a0e20abd3cb7a7712f3425ecb794c23310a468554"} Jul 18 02:46:46.770351 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:46.770305 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dng5q" podStartSLOduration=2.140567699 podStartE2EDuration="22.770290882s" totalImagesPullingTime="20.629723183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:46:46.769004631 +0000 UTC m=+398.557778401" watchObservedRunningTime="2026-07-18 02:46:46.770290882 +0000 UTC m=+398.559064651" Jul 18 02:46:46.806397 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:46.806355 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dc4b576ff-gmjwn" podStartSLOduration=5.806340241 podStartE2EDuration="5.806340241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:46:46.804165152 +0000 UTC m=+398.592938922" watchObservedRunningTime="2026-07-18 02:46:46.806340241 +0000 UTC m=+398.595114011" Jul 18 02:46:52.373641 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:52.373604 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:52.374216 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:52.373686 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:52.378519 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:52.378498 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:52.788628 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:52.788602 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dc4b576ff-gmjwn" Jul 18 02:46:52.867875 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:46:52.867845 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8475b8b55-5scrz"] Jul 18 02:47:08.286525 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.286487 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c"] Jul 18 02:47:08.487201 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.487164 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c"] Jul 18 02:47:08.487617 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.487277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.490163 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.490139 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g96m7\"" Jul 18 02:47:08.579829 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.579760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db93015-6201-403c-a36d-d5d76807ee72-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.579947 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.579823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.579947 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.579889 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.579947 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.579923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.579947 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.579945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db93015-6201-403c-a36d-d5d76807ee72-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.580130 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.579989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nct2c\" (UniqueName: \"kubernetes.io/projected/1db93015-6201-403c-a36d-d5d76807ee72-kube-api-access-nct2c\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.580130 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.580015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.580130 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.580044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.580130 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.580065 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db93015-6201-403c-a36d-d5d76807ee72-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681027 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.680994 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db93015-6201-403c-a36d-d5d76807ee72-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681178 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681178 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681178 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681178 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db93015-6201-403c-a36d-d5d76807ee72-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681445 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nct2c\" (UniqueName: \"kubernetes.io/projected/1db93015-6201-403c-a36d-d5d76807ee72-kube-api-access-nct2c\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681445 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681346 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681445 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681445 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db93015-6201-403c-a36d-d5d76807ee72-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681662 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681662 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681569 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.681662 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.681629 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.682027 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.682004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db93015-6201-403c-a36d-d5d76807ee72-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.682129 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.682111 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.683935 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.683910 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db93015-6201-403c-a36d-d5d76807ee72-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.684042 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.683945 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db93015-6201-403c-a36d-d5d76807ee72-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.690936 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.690912 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db93015-6201-403c-a36d-d5d76807ee72-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.691415 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.691373 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nct2c\" (UniqueName: \"kubernetes.io/projected/1db93015-6201-403c-a36d-d5d76807ee72-kube-api-access-nct2c\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-th65c\" (UID: \"1db93015-6201-403c-a36d-d5d76807ee72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.801979 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.801951 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g96m7\"" Jul 18 02:47:08.809317 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.809295 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:08.976299 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.976272 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c"] Jul 18 02:47:08.978035 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:47:08.978000 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db93015_6201_403c_a36d_d5d76807ee72.slice/crio-2f029f7dbad46bda1477f94a3f57cc8b13ac746560543ebbea2019cdc613b209 WatchSource:0}: Error finding container 2f029f7dbad46bda1477f94a3f57cc8b13ac746560543ebbea2019cdc613b209: Status 404 returned error can't find the container with id 2f029f7dbad46bda1477f94a3f57cc8b13ac746560543ebbea2019cdc613b209 Jul 18 02:47:08.980258 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.980224 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:47:08.980351 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.980291 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:47:08.980351 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:08.980316 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:47:09.867273 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:09.867243 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" event={"ID":"1db93015-6201-403c-a36d-d5d76807ee72","Type":"ContainerStarted","Data":"060706e4cecab08d35af1ab3d2cfe501302dcef0c621a1dfb66f8799a9ab521a"} Jul 18 02:47:09.867634 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:09.867293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" event={"ID":"1db93015-6201-403c-a36d-d5d76807ee72","Type":"ContainerStarted","Data":"2f029f7dbad46bda1477f94a3f57cc8b13ac746560543ebbea2019cdc613b209"} Jul 18 02:47:09.888838 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:09.888791 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" podStartSLOduration=1.888777632 podStartE2EDuration="1.888777632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:47:09.886985513 +0000 UTC m=+421.675759277" watchObservedRunningTime="2026-07-18 02:47:09.888777632 +0000 UTC m=+421.677551400" Jul 18 02:47:10.809959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:10.809926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:10.814669 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:10.814645 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:10.873043 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:10.873015 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:10.873765 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:10.873747 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:47:11.940021 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:11.939979 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlkl2"] Jul 18 02:47:11.996602 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:11.996573 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlkl2"] Jul 18 02:47:11.996772 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:11.996662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:11.999213 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:11.999186 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dn2vt\"" Jul 18 02:47:12.112467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:12.112440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r27x\" (UniqueName: \"kubernetes.io/projected/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90-kube-api-access-4r27x\") pod \"authorino-75d556bd77-wlkl2\" (UID: \"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90\") " pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:12.213230 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:12.213146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r27x\" (UniqueName: \"kubernetes.io/projected/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90-kube-api-access-4r27x\") pod \"authorino-75d556bd77-wlkl2\" (UID: \"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90\") " pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:12.221145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:12.221116 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r27x\" (UniqueName: \"kubernetes.io/projected/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90-kube-api-access-4r27x\") pod \"authorino-75d556bd77-wlkl2\" (UID: \"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90\") " pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:12.307082 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:12.307051 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:12.652716 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:12.652687 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlkl2"] Jul 18 02:47:12.653908 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:47:12.653881 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97b2e711_fd9b_4fcb_a30f_8d2e8754fa90.slice/crio-b3ef5b6cc2ae25af97620a7908ac3f23bb6f2a409fd1ceeefafd1e39b82fb494 WatchSource:0}: Error finding container b3ef5b6cc2ae25af97620a7908ac3f23bb6f2a409fd1ceeefafd1e39b82fb494: Status 404 returned error can't find the container with id b3ef5b6cc2ae25af97620a7908ac3f23bb6f2a409fd1ceeefafd1e39b82fb494 Jul 18 02:47:12.884729 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:12.884647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlkl2" event={"ID":"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90","Type":"ContainerStarted","Data":"b3ef5b6cc2ae25af97620a7908ac3f23bb6f2a409fd1ceeefafd1e39b82fb494"} Jul 18 02:47:15.912467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:15.912373 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlkl2" event={"ID":"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90","Type":"ContainerStarted","Data":"fdd7ec53d309531b695c9d45297b3c6cc002128746e7b4fcc529e4c49a4a1e1b"} Jul 18 02:47:15.929332 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:15.929232 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wlkl2" podStartSLOduration=2.028480755 podStartE2EDuration="4.929215509s" totalImagesPullingTime="2.900734754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:47:15.92892132 +0000 UTC m=+427.717695090" watchObservedRunningTime="2026-07-18 02:47:15.929215509 +0000 UTC m=+427.717989280" Jul 18 02:47:17.888216 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:17.888168 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8475b8b55-5scrz" podUID="3a546d2c-0853-49a5-9639-93a395bc3c53" containerName="console" containerID="cri-o://c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c" gracePeriod=15 Jul 18 02:47:18.140805 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.140752 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:47:18.271069 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271038 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpzwl\" (UniqueName: \"kubernetes.io/projected/3a546d2c-0853-49a5-9639-93a395bc3c53-kube-api-access-bpzwl\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271069 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271073 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-service-ca\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271262 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271115 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-serving-cert\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271262 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271147 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-trusted-ca-bundle\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271262 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271204 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-console-config\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271394 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271324 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-oauth-config\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271394 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271363 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-oauth-serving-cert\") pod \"3a546d2c-0853-49a5-9639-93a395bc3c53\" (UID: \"3a546d2c-0853-49a5-9639-93a395bc3c53\") " Jul 18 02:47:18.271614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271529 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:47:18.271614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271581 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-console-config" (OuterVolumeSpecName: "console-config") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:47:18.271614 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271598 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:47:18.271994 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271771 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-trusted-ca-bundle\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.271994 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271785 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-console-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.271994 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271793 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-service-ca\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.271994 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.271826 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:47:18.273395 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.273371 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:47:18.273492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.273396 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a546d2c-0853-49a5-9639-93a395bc3c53-kube-api-access-bpzwl" (OuterVolumeSpecName: "kube-api-access-bpzwl") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "kube-api-access-bpzwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:47:18.273492 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.273412 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a546d2c-0853-49a5-9639-93a395bc3c53" (UID: "3a546d2c-0853-49a5-9639-93a395bc3c53"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:47:18.372688 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.372667 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-oauth-config\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.372688 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.372689 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a546d2c-0853-49a5-9639-93a395bc3c53-oauth-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.372821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.372698 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpzwl\" (UniqueName: \"kubernetes.io/projected/3a546d2c-0853-49a5-9639-93a395bc3c53-kube-api-access-bpzwl\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.372821 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.372706 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a546d2c-0853-49a5-9639-93a395bc3c53-console-serving-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:18.928150 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.928119 2560 generic.go:356] "Generic (PLEG): container finished" podID="3a546d2c-0853-49a5-9639-93a395bc3c53" containerID="c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c" exitCode=0 Jul 18 02:47:18.928550 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.928160 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8475b8b55-5scrz" Jul 18 02:47:18.928550 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.928199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8475b8b55-5scrz" event={"ID":"3a546d2c-0853-49a5-9639-93a395bc3c53","Type":"ContainerDied","Data":"c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c"} Jul 18 02:47:18.928550 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.928235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8475b8b55-5scrz" event={"ID":"3a546d2c-0853-49a5-9639-93a395bc3c53","Type":"ContainerDied","Data":"53d19d2f91be72cfe52aacaeb494ebed77b5d3678da0e36b16b4e2c69005e3f2"} Jul 18 02:47:18.928550 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.928254 2560 scope.go:117] "RemoveContainer" containerID="c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c" Jul 18 02:47:18.937156 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.937137 2560 scope.go:117] "RemoveContainer" containerID="c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c" Jul 18 02:47:18.937403 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:47:18.937384 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c\": container with ID starting with c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c not found: ID does not exist" containerID="c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c" Jul 18 02:47:18.937462 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.937414 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c"} err="failed to get container status \"c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c\": rpc error: code = NotFound desc = could not find container \"c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c\": container with ID starting with c63a676f74270b6fe0ea34a12bce580a8f82ece9c88e35f6ab2b27954aaa9b1c not found: ID does not exist" Jul 18 02:47:18.944736 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.944714 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8475b8b55-5scrz"] Jul 18 02:47:18.949131 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:18.949105 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8475b8b55-5scrz"] Jul 18 02:47:20.757107 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:20.757067 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a546d2c-0853-49a5-9639-93a395bc3c53" path="/var/lib/kubelet/pods/3a546d2c-0853-49a5-9639-93a395bc3c53/volumes" Jul 18 02:47:46.946210 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:46.946129 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlkl2"] Jul 18 02:47:46.946625 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:46.946282 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wlkl2" podUID="97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" containerName="authorino" containerID="cri-o://fdd7ec53d309531b695c9d45297b3c6cc002128746e7b4fcc529e4c49a4a1e1b" gracePeriod=30 Jul 18 02:47:47.072188 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:47.072161 2560 generic.go:356] "Generic (PLEG): container finished" podID="97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" containerID="fdd7ec53d309531b695c9d45297b3c6cc002128746e7b4fcc529e4c49a4a1e1b" exitCode=0 Jul 18 02:47:47.072303 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:47.072200 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlkl2" event={"ID":"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90","Type":"ContainerDied","Data":"fdd7ec53d309531b695c9d45297b3c6cc002128746e7b4fcc529e4c49a4a1e1b"} Jul 18 02:47:47.194493 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:47.194471 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:47.200878 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:47.200824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r27x\" (UniqueName: \"kubernetes.io/projected/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90-kube-api-access-4r27x\") pod \"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90\" (UID: \"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90\") " Jul 18 02:47:47.202698 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:47.202674 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90-kube-api-access-4r27x" (OuterVolumeSpecName: "kube-api-access-4r27x") pod "97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" (UID: "97b2e711-fd9b-4fcb-a30f-8d2e8754fa90"). InnerVolumeSpecName "kube-api-access-4r27x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:47:47.302272 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:47.302242 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r27x\" (UniqueName: \"kubernetes.io/projected/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90-kube-api-access-4r27x\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:47:48.078387 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:48.078356 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wlkl2" Jul 18 02:47:48.078387 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:48.078366 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wlkl2" event={"ID":"97b2e711-fd9b-4fcb-a30f-8d2e8754fa90","Type":"ContainerDied","Data":"b3ef5b6cc2ae25af97620a7908ac3f23bb6f2a409fd1ceeefafd1e39b82fb494"} Jul 18 02:47:48.078908 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:48.078419 2560 scope.go:117] "RemoveContainer" containerID="fdd7ec53d309531b695c9d45297b3c6cc002128746e7b4fcc529e4c49a4a1e1b" Jul 18 02:47:48.102841 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:48.102819 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlkl2"] Jul 18 02:47:48.106428 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:48.106406 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wlkl2"] Jul 18 02:47:48.757713 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:48.757674 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" path="/var/lib/kubelet/pods/97b2e711-fd9b-4fcb-a30f-8d2e8754fa90/volumes" Jul 18 02:47:51.647264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647232 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cb659cc75-p8rlx"] Jul 18 02:47:51.647712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647555 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a546d2c-0853-49a5-9639-93a395bc3c53" containerName="console" Jul 18 02:47:51.647712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647565 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a546d2c-0853-49a5-9639-93a395bc3c53" containerName="console" Jul 18 02:47:51.647712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647580 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" containerName="authorino" Jul 18 02:47:51.647712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647586 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" containerName="authorino" Jul 18 02:47:51.647712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647646 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b2e711-fd9b-4fcb-a30f-8d2e8754fa90" containerName="authorino" Jul 18 02:47:51.647712 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.647659 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a546d2c-0853-49a5-9639-93a395bc3c53" containerName="console" Jul 18 02:47:51.650499 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.650474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.653160 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.653143 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:47:51.653271 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.653250 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kt5jv\"" Jul 18 02:47:51.653897 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.653880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:47:51.663169 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.663149 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb659cc75-p8rlx"] Jul 18 02:47:51.735810 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.735780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9lqx\" (UniqueName: \"kubernetes.io/projected/d85a8efe-f285-4e53-9adb-6c4274c43dce-kube-api-access-n9lqx\") pod \"maas-controller-6cb659cc75-p8rlx\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.736008 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.735823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d85a8efe-f285-4e53-9adb-6c4274c43dce-webhook-cert\") pod \"maas-controller-6cb659cc75-p8rlx\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.837330 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.837300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9lqx\" (UniqueName: \"kubernetes.io/projected/d85a8efe-f285-4e53-9adb-6c4274c43dce-kube-api-access-n9lqx\") pod \"maas-controller-6cb659cc75-p8rlx\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.837721 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.837360 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d85a8efe-f285-4e53-9adb-6c4274c43dce-webhook-cert\") pod \"maas-controller-6cb659cc75-p8rlx\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.839846 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.839820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d85a8efe-f285-4e53-9adb-6c4274c43dce-webhook-cert\") pod \"maas-controller-6cb659cc75-p8rlx\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.844928 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.844908 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9lqx\" (UniqueName: \"kubernetes.io/projected/d85a8efe-f285-4e53-9adb-6c4274c43dce-kube-api-access-n9lqx\") pod \"maas-controller-6cb659cc75-p8rlx\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:51.962988 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:51.962884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:52.110531 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:52.110503 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb659cc75-p8rlx"] Jul 18 02:47:52.111508 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:47:52.111481 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85a8efe_f285_4e53_9adb_6c4274c43dce.slice/crio-a36efc74597257ea50cbaf6435fefe8b3266818914fdf214c96d15c1b40c5995 WatchSource:0}: Error finding container a36efc74597257ea50cbaf6435fefe8b3266818914fdf214c96d15c1b40c5995: Status 404 returned error can't find the container with id a36efc74597257ea50cbaf6435fefe8b3266818914fdf214c96d15c1b40c5995 Jul 18 02:47:53.110467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:53.110422 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" event={"ID":"d85a8efe-f285-4e53-9adb-6c4274c43dce","Type":"ContainerStarted","Data":"a36efc74597257ea50cbaf6435fefe8b3266818914fdf214c96d15c1b40c5995"} Jul 18 02:47:55.122996 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:55.122947 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" event={"ID":"d85a8efe-f285-4e53-9adb-6c4274c43dce","Type":"ContainerStarted","Data":"2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d"} Jul 18 02:47:55.123427 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:55.123133 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:47:55.141984 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:47:55.141923 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" podStartSLOduration=1.684494916 podStartE2EDuration="4.141907981s" totalImagesPullingTime="2.457413065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:47:55.140692146 +0000 UTC m=+466.929465916" watchObservedRunningTime="2026-07-18 02:47:55.141907981 +0000 UTC m=+466.930681793" Jul 18 02:48:06.131232 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.131198 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:48:06.209817 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.209788 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cb659cc75-p8rlx"] Jul 18 02:48:06.210027 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.210005 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" podUID="d85a8efe-f285-4e53-9adb-6c4274c43dce" containerName="manager" containerID="cri-o://2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d" gracePeriod=30 Jul 18 02:48:06.451167 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.451143 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:48:06.565621 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.565599 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9lqx\" (UniqueName: \"kubernetes.io/projected/d85a8efe-f285-4e53-9adb-6c4274c43dce-kube-api-access-n9lqx\") pod \"d85a8efe-f285-4e53-9adb-6c4274c43dce\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " Jul 18 02:48:06.565746 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.565643 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d85a8efe-f285-4e53-9adb-6c4274c43dce-webhook-cert\") pod \"d85a8efe-f285-4e53-9adb-6c4274c43dce\" (UID: \"d85a8efe-f285-4e53-9adb-6c4274c43dce\") " Jul 18 02:48:06.567823 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.567794 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d85a8efe-f285-4e53-9adb-6c4274c43dce-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d85a8efe-f285-4e53-9adb-6c4274c43dce" (UID: "d85a8efe-f285-4e53-9adb-6c4274c43dce"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:48:06.567823 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.567806 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85a8efe-f285-4e53-9adb-6c4274c43dce-kube-api-access-n9lqx" (OuterVolumeSpecName: "kube-api-access-n9lqx") pod "d85a8efe-f285-4e53-9adb-6c4274c43dce" (UID: "d85a8efe-f285-4e53-9adb-6c4274c43dce"). InnerVolumeSpecName "kube-api-access-n9lqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:48:06.666149 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.666117 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d85a8efe-f285-4e53-9adb-6c4274c43dce-webhook-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:48:06.666149 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:06.666149 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9lqx\" (UniqueName: \"kubernetes.io/projected/d85a8efe-f285-4e53-9adb-6c4274c43dce-kube-api-access-n9lqx\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:48:07.185556 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.185474 2560 generic.go:356] "Generic (PLEG): container finished" podID="d85a8efe-f285-4e53-9adb-6c4274c43dce" containerID="2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d" exitCode=0 Jul 18 02:48:07.185556 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.185518 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" Jul 18 02:48:07.186049 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.185559 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" event={"ID":"d85a8efe-f285-4e53-9adb-6c4274c43dce","Type":"ContainerDied","Data":"2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d"} Jul 18 02:48:07.186049 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.185608 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb659cc75-p8rlx" event={"ID":"d85a8efe-f285-4e53-9adb-6c4274c43dce","Type":"ContainerDied","Data":"a36efc74597257ea50cbaf6435fefe8b3266818914fdf214c96d15c1b40c5995"} Jul 18 02:48:07.186049 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.185631 2560 scope.go:117] "RemoveContainer" containerID="2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d" Jul 18 02:48:07.194482 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.194466 2560 scope.go:117] "RemoveContainer" containerID="2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d" Jul 18 02:48:07.194751 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:48:07.194733 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d\": container with ID starting with 2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d not found: ID does not exist" containerID="2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d" Jul 18 02:48:07.194807 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.194763 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d"} err="failed to get container status \"2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d\": rpc error: code = NotFound desc = could not find container \"2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d\": container with ID starting with 2ae3cea082268e26ffdd1c099674abcbbb47fd0065eb15f61adefdc0a9aaeb6d not found: ID does not exist" Jul 18 02:48:07.202860 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.202839 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cb659cc75-p8rlx"] Jul 18 02:48:07.206951 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:07.206930 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cb659cc75-p8rlx"] Jul 18 02:48:08.757550 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:08.757524 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d85a8efe-f285-4e53-9adb-6c4274c43dce" path="/var/lib/kubelet/pods/d85a8efe-f285-4e53-9adb-6c4274c43dce/volumes" Jul 18 02:48:27.559195 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.559158 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xqsfl"] Jul 18 02:48:27.559789 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.559655 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85a8efe-f285-4e53-9adb-6c4274c43dce" containerName="manager" Jul 18 02:48:27.559789 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.559671 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85a8efe-f285-4e53-9adb-6c4274c43dce" containerName="manager" Jul 18 02:48:27.559789 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.559760 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d85a8efe-f285-4e53-9adb-6c4274c43dce" containerName="manager" Jul 18 02:48:27.564259 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.564234 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.566807 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.566786 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 18 02:48:27.566945 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.566831 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7hp27\"" Jul 18 02:48:27.568917 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.568896 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xqsfl"] Jul 18 02:48:27.590805 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.590779 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-vzgtk"] Jul 18 02:48:27.594206 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.594188 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.606107 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.606086 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-vzgtk"] Jul 18 02:48:27.631167 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.631145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvd2v\" (UniqueName: \"kubernetes.io/projected/8516c60e-702f-4312-8850-40dac347bfe5-kube-api-access-dvd2v\") pod \"payload-pre-processing-7677b774bb-xqsfl\" (UID: \"8516c60e-702f-4312-8850-40dac347bfe5\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.631264 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.631174 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8516c60e-702f-4312-8850-40dac347bfe5-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xqsfl\" (UID: \"8516c60e-702f-4312-8850-40dac347bfe5\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.732607 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.732568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqjcc\" (UniqueName: \"kubernetes.io/projected/f30942d9-047a-4f67-91e3-f229658a28ea-kube-api-access-qqjcc\") pod \"payload-processing-5bf47cd988-vzgtk\" (UID: \"f30942d9-047a-4f67-91e3-f229658a28ea\") " pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.732786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.732633 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30942d9-047a-4f67-91e3-f229658a28ea-plugins-config-volume\") pod \"payload-processing-5bf47cd988-vzgtk\" (UID: \"f30942d9-047a-4f67-91e3-f229658a28ea\") " pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.732786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.732695 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvd2v\" (UniqueName: \"kubernetes.io/projected/8516c60e-702f-4312-8850-40dac347bfe5-kube-api-access-dvd2v\") pod \"payload-pre-processing-7677b774bb-xqsfl\" (UID: \"8516c60e-702f-4312-8850-40dac347bfe5\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.732786 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.732723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8516c60e-702f-4312-8850-40dac347bfe5-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xqsfl\" (UID: \"8516c60e-702f-4312-8850-40dac347bfe5\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.733397 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.733380 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8516c60e-702f-4312-8850-40dac347bfe5-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xqsfl\" (UID: \"8516c60e-702f-4312-8850-40dac347bfe5\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.745169 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.745148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvd2v\" (UniqueName: \"kubernetes.io/projected/8516c60e-702f-4312-8850-40dac347bfe5-kube-api-access-dvd2v\") pod \"payload-pre-processing-7677b774bb-xqsfl\" (UID: \"8516c60e-702f-4312-8850-40dac347bfe5\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.833337 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.833263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqjcc\" (UniqueName: \"kubernetes.io/projected/f30942d9-047a-4f67-91e3-f229658a28ea-kube-api-access-qqjcc\") pod \"payload-processing-5bf47cd988-vzgtk\" (UID: \"f30942d9-047a-4f67-91e3-f229658a28ea\") " pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.833337 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.833323 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30942d9-047a-4f67-91e3-f229658a28ea-plugins-config-volume\") pod \"payload-processing-5bf47cd988-vzgtk\" (UID: \"f30942d9-047a-4f67-91e3-f229658a28ea\") " pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.833955 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.833937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30942d9-047a-4f67-91e3-f229658a28ea-plugins-config-volume\") pod \"payload-processing-5bf47cd988-vzgtk\" (UID: \"f30942d9-047a-4f67-91e3-f229658a28ea\") " pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.841294 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.841269 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqjcc\" (UniqueName: \"kubernetes.io/projected/f30942d9-047a-4f67-91e3-f229658a28ea-kube-api-access-qqjcc\") pod \"payload-processing-5bf47cd988-vzgtk\" (UID: \"f30942d9-047a-4f67-91e3-f229658a28ea\") " pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:27.876055 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.876028 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:27.904512 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:27.904490 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:28.026927 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:28.026901 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xqsfl"] Jul 18 02:48:28.029036 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:48:28.029008 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8516c60e_702f_4312_8850_40dac347bfe5.slice/crio-c1430b4c873f7171b565b780f511aee5558e67cae62c546d0b16ec1574df2eb2 WatchSource:0}: Error finding container c1430b4c873f7171b565b780f511aee5558e67cae62c546d0b16ec1574df2eb2: Status 404 returned error can't find the container with id c1430b4c873f7171b565b780f511aee5558e67cae62c546d0b16ec1574df2eb2 Jul 18 02:48:28.051428 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:28.051402 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-vzgtk"] Jul 18 02:48:28.052846 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:48:28.052821 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30942d9_047a_4f67_91e3_f229658a28ea.slice/crio-8f588c96b8e7202a1e7260ea08043a7e971e78bf348ed0a733994f3f32f6d991 WatchSource:0}: Error finding container 8f588c96b8e7202a1e7260ea08043a7e971e78bf348ed0a733994f3f32f6d991: Status 404 returned error can't find the container with id 8f588c96b8e7202a1e7260ea08043a7e971e78bf348ed0a733994f3f32f6d991 Jul 18 02:48:28.302073 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:28.302036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" event={"ID":"f30942d9-047a-4f67-91e3-f229658a28ea","Type":"ContainerStarted","Data":"8f588c96b8e7202a1e7260ea08043a7e971e78bf348ed0a733994f3f32f6d991"} Jul 18 02:48:28.303225 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:28.303201 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" event={"ID":"8516c60e-702f-4312-8850-40dac347bfe5","Type":"ContainerStarted","Data":"c1430b4c873f7171b565b780f511aee5558e67cae62c546d0b16ec1574df2eb2"} Jul 18 02:48:30.317711 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:30.317682 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" event={"ID":"8516c60e-702f-4312-8850-40dac347bfe5","Type":"ContainerStarted","Data":"76481174f940966cf2316d730dd8b891a72a4de0c842c54a48003e37ffd35b62"} Jul 18 02:48:30.318192 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:30.317829 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:30.319204 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:30.319179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" event={"ID":"f30942d9-047a-4f67-91e3-f229658a28ea","Type":"ContainerStarted","Data":"3dee36a448c9ca54b7df668166062a7b3c18018d2136d355e10a823f07bf81cd"} Jul 18 02:48:30.319337 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:30.319320 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:48:30.337495 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:30.337443 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" podStartSLOduration=1.1904592489999999 podStartE2EDuration="3.337427064s" totalImagesPullingTime="2.146967815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:48:30.332815224 +0000 UTC m=+502.121588997" watchObservedRunningTime="2026-07-18 02:48:30.337427064 +0000 UTC m=+502.126200835" Jul 18 02:48:30.354239 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:30.354171 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" podStartSLOduration=1.2238022769999999 podStartE2EDuration="3.354152241s" totalImagesPullingTime="2.130349964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:48:30.352850884 +0000 UTC m=+502.141624654" watchObservedRunningTime="2026-07-18 02:48:30.354152241 +0000 UTC m=+502.142926012" Jul 18 02:48:35.347495 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:35.347428 2560 generic.go:356] "Generic (PLEG): container finished" podID="1db93015-6201-403c-a36d-d5d76807ee72" containerID="060706e4cecab08d35af1ab3d2cfe501302dcef0c621a1dfb66f8799a9ab521a" exitCode=0 Jul 18 02:48:35.347495 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:35.347464 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" event={"ID":"1db93015-6201-403c-a36d-d5d76807ee72","Type":"ContainerDied","Data":"060706e4cecab08d35af1ab3d2cfe501302dcef0c621a1dfb66f8799a9ab521a"} Jul 18 02:48:35.347896 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:35.347800 2560 scope.go:117] "RemoveContainer" containerID="060706e4cecab08d35af1ab3d2cfe501302dcef0c621a1dfb66f8799a9ab521a" Jul 18 02:48:35.348736 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:35.348707 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:48:35.348790 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:35.348777 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:48:35.348841 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:35.348829 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:48:36.358247 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:36.358199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" event={"ID":"1db93015-6201-403c-a36d-d5d76807ee72","Type":"ContainerStarted","Data":"75f7a7b970f984511c8e107bd9888e0a4be05699859e0b92ae2ae913071e5e27"} Jul 18 02:48:36.809794 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:36.809761 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:48:36.810837 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:36.810817 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-th65c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 18 02:48:36.810895 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:36.810862 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" podUID="1db93015-6201-403c-a36d-d5d76807ee72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 18 02:48:37.810327 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:37.810294 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-th65c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 18 02:48:37.810720 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:37.810346 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" podUID="1db93015-6201-403c-a36d-d5d76807ee72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 18 02:48:38.810670 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:38.810636 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-th65c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 18 02:48:38.811090 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:38.810685 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" podUID="1db93015-6201-403c-a36d-d5d76807ee72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 18 02:48:39.809934 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:39.809896 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-th65c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 18 02:48:39.810150 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:39.809954 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" podUID="1db93015-6201-403c-a36d-d5d76807ee72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 18 02:48:40.814161 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:40.814128 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:48:40.814536 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:40.814373 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:48:40.815191 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:40.815173 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-th65c" Jul 18 02:48:41.325769 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:41.325731 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xqsfl" Jul 18 02:48:41.325927 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:48:41.325795 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-vzgtk" Jul 18 02:49:17.088863 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.088779 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk"] Jul 18 02:49:17.093146 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.093124 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.097024 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.096999 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 18 02:49:17.097220 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.097201 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 02:49:17.097314 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.097209 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 02:49:17.097314 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.097233 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j4nw7\"" Jul 18 02:49:17.105451 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.105429 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk"] Jul 18 02:49:17.164769 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.164727 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.164914 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.164787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a4852d8-442b-4f69-bb30-967abc080b18-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.164914 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.164845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.164914 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.164864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.164914 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.164885 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnxwk\" (UniqueName: \"kubernetes.io/projected/8a4852d8-442b-4f69-bb30-967abc080b18-kube-api-access-wnxwk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.164914 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.164903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.265688 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.265657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.265688 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.265691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.265921 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.265709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnxwk\" (UniqueName: \"kubernetes.io/projected/8a4852d8-442b-4f69-bb30-967abc080b18-kube-api-access-wnxwk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.265921 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.265838 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.266067 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.265953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.266067 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.266024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a4852d8-442b-4f69-bb30-967abc080b18-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.266180 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.266143 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.266236 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.266201 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.266236 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.266215 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.268251 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.268225 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a4852d8-442b-4f69-bb30-967abc080b18-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.268425 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.268408 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a4852d8-442b-4f69-bb30-967abc080b18-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.273126 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.273104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnxwk\" (UniqueName: \"kubernetes.io/projected/8a4852d8-442b-4f69-bb30-967abc080b18-kube-api-access-wnxwk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ngfxk\" (UID: \"8a4852d8-442b-4f69-bb30-967abc080b18\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.403691 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.403615 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:17.534771 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.534740 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk"] Jul 18 02:49:17.536260 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:49:17.536235 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a4852d8_442b_4f69_bb30_967abc080b18.slice/crio-5483d5f508c90c890d85480fe26bb309b3b8b6008fb26c4fd2c4da40a48e0bde WatchSource:0}: Error finding container 5483d5f508c90c890d85480fe26bb309b3b8b6008fb26c4fd2c4da40a48e0bde: Status 404 returned error can't find the container with id 5483d5f508c90c890d85480fe26bb309b3b8b6008fb26c4fd2c4da40a48e0bde Jul 18 02:49:17.574107 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:17.574076 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" event={"ID":"8a4852d8-442b-4f69-bb30-967abc080b18","Type":"ContainerStarted","Data":"5483d5f508c90c890d85480fe26bb309b3b8b6008fb26c4fd2c4da40a48e0bde"} Jul 18 02:49:19.586150 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:19.586111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" event={"ID":"8a4852d8-442b-4f69-bb30-967abc080b18","Type":"ContainerStarted","Data":"4f02d3e2c60eed34fc6d3f0942d72ae5aa11e49072b82f54fb9c7a92571b32f2"} Jul 18 02:49:19.604848 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:19.604793 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" podStartSLOduration=1.611711551 podStartE2EDuration="2.604777525s" totalImagesPullingTime="993.065974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:49:19.602752437 +0000 UTC m=+551.391526209" watchObservedRunningTime="2026-07-18 02:49:19.604777525 +0000 UTC m=+551.393551295" Jul 18 02:49:22.688367 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.688337 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h"] Jul 18 02:49:22.691959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.691939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.694755 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.694736 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 18 02:49:22.701684 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.701662 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h"] Jul 18 02:49:22.819467 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.819434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.819650 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.819475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.819650 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.819545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.819650 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.819568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cf117c-214b-43e0-84f3-b39df8358b7c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.819814 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.819673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f5l4\" (UniqueName: \"kubernetes.io/projected/e4cf117c-214b-43e0-84f3-b39df8358b7c-kube-api-access-8f5l4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.819814 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.819703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.920841 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.920807 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f5l4\" (UniqueName: \"kubernetes.io/projected/e4cf117c-214b-43e0-84f3-b39df8358b7c-kube-api-access-8f5l4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.920841 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.920843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921117 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.920988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921117 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.921029 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921117 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.921079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921117 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.921095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cf117c-214b-43e0-84f3-b39df8358b7c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921306 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.921209 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921409 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.921392 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.921464 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.921442 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.923299 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.923274 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cf117c-214b-43e0-84f3-b39df8358b7c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.923457 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.923440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cf117c-214b-43e0-84f3-b39df8358b7c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:22.929145 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:22.929124 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f5l4\" (UniqueName: \"kubernetes.io/projected/e4cf117c-214b-43e0-84f3-b39df8358b7c-kube-api-access-8f5l4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h\" (UID: \"e4cf117c-214b-43e0-84f3-b39df8358b7c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:23.003319 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:23.003262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:23.136420 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:23.136395 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h"] Jul 18 02:49:23.137620 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:49:23.137593 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cf117c_214b_43e0_84f3_b39df8358b7c.slice/crio-c14fc35e76201c795764b7f47dafeae2a01a6c3c5cf04afc19e475fd7158426f WatchSource:0}: Error finding container c14fc35e76201c795764b7f47dafeae2a01a6c3c5cf04afc19e475fd7158426f: Status 404 returned error can't find the container with id c14fc35e76201c795764b7f47dafeae2a01a6c3c5cf04afc19e475fd7158426f Jul 18 02:49:23.608725 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:23.608626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" event={"ID":"e4cf117c-214b-43e0-84f3-b39df8358b7c","Type":"ContainerStarted","Data":"6c61ff8bcad9518de2d4c30a3bcdce9b6d816c592dc1fe41406ac34717664dee"} Jul 18 02:49:23.608725 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:23.608687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" event={"ID":"e4cf117c-214b-43e0-84f3-b39df8358b7c","Type":"ContainerStarted","Data":"c14fc35e76201c795764b7f47dafeae2a01a6c3c5cf04afc19e475fd7158426f"} Jul 18 02:49:23.626909 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:23.626840 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" podStartSLOduration=1.46524823 podStartE2EDuration="1.626806232s" totalImagesPullingTime="161.558002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:49:23.626731439 +0000 UTC m=+555.415505210" watchObservedRunningTime="2026-07-18 02:49:23.626806232 +0000 UTC m=+555.415580003" Jul 18 02:49:27.403959 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:27.403926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:27.404365 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:27.403995 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:27.416744 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:27.416718 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:27.640814 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:27.640789 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ngfxk" Jul 18 02:49:33.003984 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:33.003932 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:33.004403 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:33.004011 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:33.016294 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:33.016267 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:49:33.674612 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:49:33.674583 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h" Jul 18 02:50:08.658423 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:50:08.658395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:50:08.659653 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:50:08.659629 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:52:52.097517 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.097481 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6d58f489-wbgpf"] Jul 18 02:52:52.101539 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.101518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.105170 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.105145 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:52:52.105322 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.105195 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:52:52.105322 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.105156 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kt5jv\"" Jul 18 02:52:52.111195 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.111172 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6d58f489-wbgpf"] Jul 18 02:52:52.248871 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.248835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a198193d-b36b-4747-8bd8-8956fd4f59c7-webhook-cert\") pod \"maas-controller-d6d58f489-wbgpf\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.248871 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.248872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67hqm\" (UniqueName: \"kubernetes.io/projected/a198193d-b36b-4747-8bd8-8956fd4f59c7-kube-api-access-67hqm\") pod \"maas-controller-d6d58f489-wbgpf\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.349864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.349756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a198193d-b36b-4747-8bd8-8956fd4f59c7-webhook-cert\") pod \"maas-controller-d6d58f489-wbgpf\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.349864 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.349813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67hqm\" (UniqueName: \"kubernetes.io/projected/a198193d-b36b-4747-8bd8-8956fd4f59c7-kube-api-access-67hqm\") pod \"maas-controller-d6d58f489-wbgpf\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.352432 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.352406 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a198193d-b36b-4747-8bd8-8956fd4f59c7-webhook-cert\") pod \"maas-controller-d6d58f489-wbgpf\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.357865 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.357843 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67hqm\" (UniqueName: \"kubernetes.io/projected/a198193d-b36b-4747-8bd8-8956fd4f59c7-kube-api-access-67hqm\") pod \"maas-controller-d6d58f489-wbgpf\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.414529 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.414502 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:52.542003 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.541982 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6d58f489-wbgpf"] Jul 18 02:52:52.543378 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:52:52.543356 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda198193d_b36b_4747_8bd8_8956fd4f59c7.slice/crio-98e406e9565823e68af88b0ddba96bc7f18ef4d974316f6ea681197263beff5d WatchSource:0}: Error finding container 98e406e9565823e68af88b0ddba96bc7f18ef4d974316f6ea681197263beff5d: Status 404 returned error can't find the container with id 98e406e9565823e68af88b0ddba96bc7f18ef4d974316f6ea681197263beff5d Jul 18 02:52:52.544651 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.544634 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:52:52.702177 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:52.702114 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6d58f489-wbgpf" event={"ID":"a198193d-b36b-4747-8bd8-8956fd4f59c7","Type":"ContainerStarted","Data":"98e406e9565823e68af88b0ddba96bc7f18ef4d974316f6ea681197263beff5d"} Jul 18 02:52:53.709631 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:53.709599 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6d58f489-wbgpf" event={"ID":"a198193d-b36b-4747-8bd8-8956fd4f59c7","Type":"ContainerStarted","Data":"6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6"} Jul 18 02:52:53.710039 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:53.709707 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:52:53.734306 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:52:53.734258 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6d58f489-wbgpf" podStartSLOduration=1.217866962 podStartE2EDuration="1.734245689s" totalImagesPullingTime="516.378727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:52:53.731567377 +0000 UTC m=+765.520341153" watchObservedRunningTime="2026-07-18 02:52:53.734245689 +0000 UTC m=+765.523019479" Jul 18 02:53:04.717961 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:04.717930 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:53:09.377709 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.377668 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6d58f489-wbgpf"] Jul 18 02:53:09.378193 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.377902 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d6d58f489-wbgpf" podUID="a198193d-b36b-4747-8bd8-8956fd4f59c7" containerName="manager" containerID="cri-o://6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6" gracePeriod=30 Jul 18 02:53:09.624153 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.624131 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:53:09.696306 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.696220 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a198193d-b36b-4747-8bd8-8956fd4f59c7-webhook-cert\") pod \"a198193d-b36b-4747-8bd8-8956fd4f59c7\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " Jul 18 02:53:09.696306 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.696295 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67hqm\" (UniqueName: \"kubernetes.io/projected/a198193d-b36b-4747-8bd8-8956fd4f59c7-kube-api-access-67hqm\") pod \"a198193d-b36b-4747-8bd8-8956fd4f59c7\" (UID: \"a198193d-b36b-4747-8bd8-8956fd4f59c7\") " Jul 18 02:53:09.698189 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.698160 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a198193d-b36b-4747-8bd8-8956fd4f59c7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a198193d-b36b-4747-8bd8-8956fd4f59c7" (UID: "a198193d-b36b-4747-8bd8-8956fd4f59c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:53:09.698189 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.698165 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a198193d-b36b-4747-8bd8-8956fd4f59c7-kube-api-access-67hqm" (OuterVolumeSpecName: "kube-api-access-67hqm") pod "a198193d-b36b-4747-8bd8-8956fd4f59c7" (UID: "a198193d-b36b-4747-8bd8-8956fd4f59c7"). InnerVolumeSpecName "kube-api-access-67hqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:53:09.794486 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.794452 2560 generic.go:356] "Generic (PLEG): container finished" podID="a198193d-b36b-4747-8bd8-8956fd4f59c7" containerID="6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6" exitCode=0 Jul 18 02:53:09.794655 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.794497 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6d58f489-wbgpf" Jul 18 02:53:09.794655 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.794500 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6d58f489-wbgpf" event={"ID":"a198193d-b36b-4747-8bd8-8956fd4f59c7","Type":"ContainerDied","Data":"6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6"} Jul 18 02:53:09.794655 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.794626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6d58f489-wbgpf" event={"ID":"a198193d-b36b-4747-8bd8-8956fd4f59c7","Type":"ContainerDied","Data":"98e406e9565823e68af88b0ddba96bc7f18ef4d974316f6ea681197263beff5d"} Jul 18 02:53:09.794655 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.794650 2560 scope.go:117] "RemoveContainer" containerID="6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6" Jul 18 02:53:09.797452 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.797430 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67hqm\" (UniqueName: \"kubernetes.io/projected/a198193d-b36b-4747-8bd8-8956fd4f59c7-kube-api-access-67hqm\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:53:09.797552 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.797458 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a198193d-b36b-4747-8bd8-8956fd4f59c7-webhook-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 02:53:09.804279 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.804261 2560 scope.go:117] "RemoveContainer" containerID="6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6" Jul 18 02:53:09.804534 ip-10-0-138-139 kubenswrapper[2560]: E0718 02:53:09.804516 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6\": container with ID starting with 6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6 not found: ID does not exist" containerID="6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6" Jul 18 02:53:09.804582 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.804541 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6"} err="failed to get container status \"6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6\": rpc error: code = NotFound desc = could not find container \"6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6\": container with ID starting with 6b70a6c6debe6b4df45c07c1d28741475b299d3417718d2b96f830c07e7c0bb6 not found: ID does not exist" Jul 18 02:53:09.818259 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.818229 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6d58f489-wbgpf"] Jul 18 02:53:09.821855 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:09.821832 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d6d58f489-wbgpf"] Jul 18 02:53:10.647355 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.647317 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6d58f489-m6grn"] Jul 18 02:53:10.647768 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.647739 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a198193d-b36b-4747-8bd8-8956fd4f59c7" containerName="manager" Jul 18 02:53:10.647768 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.647760 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a198193d-b36b-4747-8bd8-8956fd4f59c7" containerName="manager" Jul 18 02:53:10.647881 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.647868 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a198193d-b36b-4747-8bd8-8956fd4f59c7" containerName="manager" Jul 18 02:53:10.652278 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.652257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.655923 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.655899 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:53:10.656904 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.656886 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:53:10.657030 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.656887 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kt5jv\"" Jul 18 02:53:10.666865 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.666844 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6d58f489-m6grn"] Jul 18 02:53:10.704479 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.704454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1284e775-8b46-4e7d-aa84-3b59ebaa1de8-webhook-cert\") pod \"maas-controller-d6d58f489-m6grn\" (UID: \"1284e775-8b46-4e7d-aa84-3b59ebaa1de8\") " pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.704579 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.704501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25jz6\" (UniqueName: \"kubernetes.io/projected/1284e775-8b46-4e7d-aa84-3b59ebaa1de8-kube-api-access-25jz6\") pod \"maas-controller-d6d58f489-m6grn\" (UID: \"1284e775-8b46-4e7d-aa84-3b59ebaa1de8\") " pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.757816 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.757789 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a198193d-b36b-4747-8bd8-8956fd4f59c7" path="/var/lib/kubelet/pods/a198193d-b36b-4747-8bd8-8956fd4f59c7/volumes" Jul 18 02:53:10.805435 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.805411 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1284e775-8b46-4e7d-aa84-3b59ebaa1de8-webhook-cert\") pod \"maas-controller-d6d58f489-m6grn\" (UID: \"1284e775-8b46-4e7d-aa84-3b59ebaa1de8\") " pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.805532 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.805496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25jz6\" (UniqueName: \"kubernetes.io/projected/1284e775-8b46-4e7d-aa84-3b59ebaa1de8-kube-api-access-25jz6\") pod \"maas-controller-d6d58f489-m6grn\" (UID: \"1284e775-8b46-4e7d-aa84-3b59ebaa1de8\") " pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.807653 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.807636 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1284e775-8b46-4e7d-aa84-3b59ebaa1de8-webhook-cert\") pod \"maas-controller-d6d58f489-m6grn\" (UID: \"1284e775-8b46-4e7d-aa84-3b59ebaa1de8\") " pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.822703 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.822683 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25jz6\" (UniqueName: \"kubernetes.io/projected/1284e775-8b46-4e7d-aa84-3b59ebaa1de8-kube-api-access-25jz6\") pod \"maas-controller-d6d58f489-m6grn\" (UID: \"1284e775-8b46-4e7d-aa84-3b59ebaa1de8\") " pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:10.963746 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:10.963685 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:11.097597 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:11.097569 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6d58f489-m6grn"] Jul 18 02:53:11.099586 ip-10-0-138-139 kubenswrapper[2560]: W0718 02:53:11.099549 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1284e775_8b46_4e7d_aa84_3b59ebaa1de8.slice/crio-a1ddb9e09742fb14921700bfaf69464dc1111443279f91ee7620a7ad3531d68a WatchSource:0}: Error finding container a1ddb9e09742fb14921700bfaf69464dc1111443279f91ee7620a7ad3531d68a: Status 404 returned error can't find the container with id a1ddb9e09742fb14921700bfaf69464dc1111443279f91ee7620a7ad3531d68a Jul 18 02:53:11.806378 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:11.806340 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6d58f489-m6grn" event={"ID":"1284e775-8b46-4e7d-aa84-3b59ebaa1de8","Type":"ContainerStarted","Data":"865bc136160d80b4865cc95aa567b632447794deee4f50d81404e3d79d9ff65c"} Jul 18 02:53:11.806763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:11.806426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6d58f489-m6grn" event={"ID":"1284e775-8b46-4e7d-aa84-3b59ebaa1de8","Type":"ContainerStarted","Data":"a1ddb9e09742fb14921700bfaf69464dc1111443279f91ee7620a7ad3531d68a"} Jul 18 02:53:11.806763 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:11.806457 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:53:11.830163 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:11.830110 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6d58f489-m6grn" podStartSLOduration=1.54893915 podStartE2EDuration="1.830096269s" totalImagesPullingTime="281.157119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:53:11.828005096 +0000 UTC m=+783.616778865" watchObservedRunningTime="2026-07-18 02:53:11.830096269 +0000 UTC m=+783.618870039" Jul 18 02:53:22.813088 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:53:22.813058 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6d58f489-m6grn" Jul 18 02:55:08.698612 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:55:08.698574 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 02:55:08.702452 ip-10-0-138-139 kubenswrapper[2560]: I0718 02:55:08.702427 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:00:00.195290 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.195209 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx"] Jul 18 03:00:00.198720 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.198704 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" Jul 18 03:00:00.205648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.205627 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dghrk\"" Jul 18 03:00:00.208017 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.207998 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" Jul 18 03:00:00.214270 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.214248 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx"] Jul 18 03:00:00.344251 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.344223 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx"] Jul 18 03:00:00.345045 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:00:00.345020 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8acdb97_cf72_4bfe_aea3_57482a1068a8.slice/crio-5a865c62965529a0bb70200758db02d8735e78b947c6e1832057412bdc9d354e WatchSource:0}: Error finding container 5a865c62965529a0bb70200758db02d8735e78b947c6e1832057412bdc9d354e: Status 404 returned error can't find the container with id 5a865c62965529a0bb70200758db02d8735e78b947c6e1832057412bdc9d354e Jul 18 03:00:00.347086 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.347069 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 03:00:00.960390 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:00.960354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" event={"ID":"c8acdb97-cf72-4bfe-aea3-57482a1068a8","Type":"ContainerStarted","Data":"5a865c62965529a0bb70200758db02d8735e78b947c6e1832057412bdc9d354e"} Jul 18 03:00:08.737921 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:08.737895 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:00:08.743863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:08.743841 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:00:16.047736 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:16.047645 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" event={"ID":"c8acdb97-cf72-4bfe-aea3-57482a1068a8","Type":"ContainerStarted","Data":"73f2d04485e4438bb9d4a2f95a1434d971e1c2e6aa84701cf0496924bc8cd67a"} Jul 18 03:00:16.064320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:16.064274 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" podStartSLOduration=0.621042032 podStartE2EDuration="16.064260224s" totalImagesPullingTime="15.443218192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:00:16.062359577 +0000 UTC m=+1207.851133347" watchObservedRunningTime="2026-07-18 03:00:16.064260224 +0000 UTC m=+1207.853033998" Jul 18 03:00:17.054309 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:17.054272 2560 generic.go:356] "Generic (PLEG): container finished" podID="c8acdb97-cf72-4bfe-aea3-57482a1068a8" containerID="73f2d04485e4438bb9d4a2f95a1434d971e1c2e6aa84701cf0496924bc8cd67a" exitCode=0 Jul 18 03:00:17.054676 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:17.054350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" event={"ID":"c8acdb97-cf72-4bfe-aea3-57482a1068a8","Type":"ContainerDied","Data":"73f2d04485e4438bb9d4a2f95a1434d971e1c2e6aa84701cf0496924bc8cd67a"} Jul 18 03:00:18.197432 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:18.197400 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" Jul 18 03:00:19.065930 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:19.065838 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" Jul 18 03:00:19.065930 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:19.065863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739060-sj7mx" event={"ID":"c8acdb97-cf72-4bfe-aea3-57482a1068a8","Type":"ContainerDied","Data":"5a865c62965529a0bb70200758db02d8735e78b947c6e1832057412bdc9d354e"} Jul 18 03:00:19.065930 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:00:19.065920 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a865c62965529a0bb70200758db02d8735e78b947c6e1832057412bdc9d354e" Jul 18 03:05:08.773341 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:08.773308 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:05:08.780905 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:08.780887 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:05:16.315555 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.315518 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp"] Jul 18 03:05:16.316029 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.315925 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8acdb97-cf72-4bfe-aea3-57482a1068a8" containerName="cleanup" Jul 18 03:05:16.316029 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.315937 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8acdb97-cf72-4bfe-aea3-57482a1068a8" containerName="cleanup" Jul 18 03:05:16.316156 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.316037 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8acdb97-cf72-4bfe-aea3-57482a1068a8" containerName="cleanup" Jul 18 03:05:16.319192 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.319174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.323636 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.323602 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-43e0ec51-openshift-default-dockercfg-pkkd7\"" Jul 18 03:05:16.346869 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.346843 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp"] Jul 18 03:05:16.395162 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-token\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395260 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-socket\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395260 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istiod-ca-cert\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395335 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395266 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-podinfo\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395335 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-certs\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395440 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-data\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395478 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-envoy\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395526 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-credential-socket\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.395576 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.395525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h95xc\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-kube-api-access-h95xc\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496389 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-credential-socket\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496389 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h95xc\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-kube-api-access-h95xc\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-token\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-socket\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istiod-ca-cert\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496549 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-podinfo\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-certs\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-data\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.496647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496638 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-envoy\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.497049 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.496793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-credential-socket\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.497110 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.497074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-certs\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.497295 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.497259 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-data\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.497454 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.497339 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istiod-ca-cert\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.497553 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.497448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-socket\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.499146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.499119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-envoy\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.499653 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.499636 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-podinfo\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.504919 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.504894 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h95xc\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-kube-api-access-h95xc\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.505030 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.504940 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-token\") pod \"e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.630205 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.630152 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:16.766287 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.766258 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp"] Jul 18 03:05:16.766940 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:05:16.766910 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4c62f6_c833_4724_b024_387d5b1cf5dd.slice/crio-5e1ca7e8807cd55eaf24dc6e6e10abcedb380b9fe293aad54072c59b21b5268a WatchSource:0}: Error finding container 5e1ca7e8807cd55eaf24dc6e6e10abcedb380b9fe293aad54072c59b21b5268a: Status 404 returned error can't find the container with id 5e1ca7e8807cd55eaf24dc6e6e10abcedb380b9fe293aad54072c59b21b5268a Jul 18 03:05:16.768931 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.768904 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:05:16.769020 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.768961 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:05:16.769065 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:16.769034 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:05:17.682008 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:17.681951 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" event={"ID":"8b4c62f6-c833-4724-b024-387d5b1cf5dd","Type":"ContainerStarted","Data":"633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76"} Jul 18 03:05:17.682595 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:17.682029 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" event={"ID":"8b4c62f6-c833-4724-b024-387d5b1cf5dd","Type":"ContainerStarted","Data":"5e1ca7e8807cd55eaf24dc6e6e10abcedb380b9fe293aad54072c59b21b5268a"} Jul 18 03:05:17.704083 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:17.704037 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" podStartSLOduration=1.704023144 podStartE2EDuration="1.704023144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:05:17.702214191 +0000 UTC m=+1509.490987960" watchObservedRunningTime="2026-07-18 03:05:17.704023144 +0000 UTC m=+1509.492796914" Jul 18 03:05:18.630958 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:18.630920 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:18.635589 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:18.635564 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:18.687609 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:18.687586 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:18.688275 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:18.688259 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:31.767498 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:31.767468 2560 generic.go:356] "Generic (PLEG): container finished" podID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerID="633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76" exitCode=0 Jul 18 03:05:31.767921 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:31.767539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" event={"ID":"8b4c62f6-c833-4724-b024-387d5b1cf5dd","Type":"ContainerDied","Data":"633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76"} Jul 18 03:05:31.767921 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:31.767837 2560 scope.go:117] "RemoveContainer" containerID="633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76" Jul 18 03:05:31.768719 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:31.768692 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:05:31.768806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:31.768757 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:05:31.768806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:31.768787 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:05:32.773872 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:32.773837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" event={"ID":"8b4c62f6-c833-4724-b024-387d5b1cf5dd","Type":"ContainerStarted","Data":"4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039"} Jul 18 03:05:33.631073 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:33.631038 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:33.632437 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:33.632399 2560 patch_prober.go:28] interesting pod/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 18 03:05:33.632571 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:33.632471 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 18 03:05:33.687463 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:33.687437 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:34.631303 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:34.631269 2560 patch_prober.go:28] interesting pod/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 18 03:05:34.631889 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:34.631857 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 18 03:05:35.631160 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:35.631124 2560 patch_prober.go:28] interesting pod/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 18 03:05:35.631358 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:35.631186 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 18 03:05:36.634583 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:36.634546 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:36.635580 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:36.635562 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:05:59.841927 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:59.841895 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp"] Jul 18 03:05:59.842459 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:05:59.842131 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" containerID="cri-o://4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039" gracePeriod=30 Jul 18 03:06:00.856379 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.856347 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v"] Jul 18 03:06:00.859986 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.859942 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.862543 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.862521 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a70f0b50-gw-openshift-default-dockercfg-jsf2g\"" Jul 18 03:06:00.871514 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.871489 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v"] Jul 18 03:06:00.984564 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984532 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-credential-socket\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984564 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-socket\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-podinfo\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k72s5\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-kube-api-access-k72s5\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984739 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-envoy\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-token\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984955 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-data\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984955 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984840 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a6680a-817b-4c46-97fc-fc748dbe226e-istiod-ca-cert\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:00.984955 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:00.984898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-certs\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.085938 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.085911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-podinfo\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.085945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k72s5\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-kube-api-access-k72s5\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.085964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-envoy\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086013 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-token\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086035 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-data\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a6680a-817b-4c46-97fc-fc748dbe226e-istiod-ca-cert\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-certs\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086438 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-credential-socket\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086438 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-socket\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086543 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086507 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-certs\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086598 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086537 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-data\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086678 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086658 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-credential-socket\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086807 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086758 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-socket\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.086897 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.086880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a6680a-817b-4c46-97fc-fc748dbe226e-istiod-ca-cert\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.088757 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.088730 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-envoy\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.088904 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.088887 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-podinfo\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.094318 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.094299 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-token\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.094733 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.094714 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k72s5\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-kube-api-access-k72s5\") pod \"e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.171730 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.171671 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:01.307306 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.307279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v"] Jul 18 03:06:01.308709 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:06:01.308676 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a6680a_817b_4c46_97fc_fc748dbe226e.slice/crio-1b64c9c0f0d29ced4aadde9e798636aa0c04ac471908657166de60c36e0de25b WatchSource:0}: Error finding container 1b64c9c0f0d29ced4aadde9e798636aa0c04ac471908657166de60c36e0de25b: Status 404 returned error can't find the container with id 1b64c9c0f0d29ced4aadde9e798636aa0c04ac471908657166de60c36e0de25b Jul 18 03:06:01.311452 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.311411 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:01.311530 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.311504 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:01.311573 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.311545 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:01.931353 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.931318 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" event={"ID":"57a6680a-817b-4c46-97fc-fc748dbe226e","Type":"ContainerStarted","Data":"35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929"} Jul 18 03:06:01.931727 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.931369 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" event={"ID":"57a6680a-817b-4c46-97fc-fc748dbe226e","Type":"ContainerStarted","Data":"1b64c9c0f0d29ced4aadde9e798636aa0c04ac471908657166de60c36e0de25b"} Jul 18 03:06:01.952788 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:01.952746 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podStartSLOduration=1.95273119 podStartE2EDuration="1.95273119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:06:01.951117839 +0000 UTC m=+1553.739891612" watchObservedRunningTime="2026-07-18 03:06:01.95273119 +0000 UTC m=+1553.741504961" Jul 18 03:06:02.172266 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:02.172235 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:02.176946 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:02.176924 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:02.936631 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:02.936588 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:02.937552 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:02.937527 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:05.093885 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.093862 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:06:05.226006 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.225922 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-data\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226006 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.225964 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-podinfo\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226006 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226001 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h95xc\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-kube-api-access-h95xc\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226047 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-socket\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226127 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-token\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226155 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-envoy\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226192 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-certs\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226239 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istiod-ca-cert\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226496 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226266 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-credential-socket\") pod \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\" (UID: \"8b4c62f6-c833-4724-b024-387d5b1cf5dd\") " Jul 18 03:06:05.226496 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226381 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:05.226608 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226528 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:05.226608 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226560 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:05.226720 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226625 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.226720 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226641 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.226720 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226656 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.226876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226730 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:05.226876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.226804 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:06:05.228309 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.228285 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-kube-api-access-h95xc" (OuterVolumeSpecName: "kube-api-access-h95xc") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "kube-api-access-h95xc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:06:05.228403 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.228331 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:06:05.228519 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.228500 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:06:05.228519 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.228511 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b4c62f6-c833-4724-b024-387d5b1cf5dd" (UID: "8b4c62f6-c833-4724-b024-387d5b1cf5dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:05.328036 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.328013 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.328036 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.328034 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.328183 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.328048 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.328183 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.328058 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.328183 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.328066 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b4c62f6-c833-4724-b024-387d5b1cf5dd-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.328183 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.328074 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h95xc\" (UniqueName: \"kubernetes.io/projected/8b4c62f6-c833-4724-b024-387d5b1cf5dd-kube-api-access-h95xc\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:05.955742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.955708 2560 generic.go:356] "Generic (PLEG): container finished" podID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerID="4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039" exitCode=0 Jul 18 03:06:05.955933 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.955757 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" Jul 18 03:06:05.955933 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.955788 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" event={"ID":"8b4c62f6-c833-4724-b024-387d5b1cf5dd","Type":"ContainerDied","Data":"4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039"} Jul 18 03:06:05.955933 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.955843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp" event={"ID":"8b4c62f6-c833-4724-b024-387d5b1cf5dd","Type":"ContainerDied","Data":"5e1ca7e8807cd55eaf24dc6e6e10abcedb380b9fe293aad54072c59b21b5268a"} Jul 18 03:06:05.955933 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.955868 2560 scope.go:117] "RemoveContainer" containerID="4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039" Jul 18 03:06:05.965076 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.964870 2560 scope.go:117] "RemoveContainer" containerID="633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76" Jul 18 03:06:05.973332 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.973315 2560 scope.go:117] "RemoveContainer" containerID="4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039" Jul 18 03:06:05.973586 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:06:05.973553 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039\": container with ID starting with 4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039 not found: ID does not exist" containerID="4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039" Jul 18 03:06:05.973696 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.973590 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039"} err="failed to get container status \"4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039\": rpc error: code = NotFound desc = could not find container \"4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039\": container with ID starting with 4deb821cdf8cab1079e8750c93f62b40a2409a3377e993e9f7cea03e54dba039 not found: ID does not exist" Jul 18 03:06:05.973696 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.973609 2560 scope.go:117] "RemoveContainer" containerID="633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76" Jul 18 03:06:05.973829 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:06:05.973810 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76\": container with ID starting with 633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76 not found: ID does not exist" containerID="633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76" Jul 18 03:06:05.973873 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.973835 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76"} err="failed to get container status \"633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76\": rpc error: code = NotFound desc = could not find container \"633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76\": container with ID starting with 633b6d7bb59459c41c1bc884dc205e354061ec648b4c810642210057cac30d76 not found: ID does not exist" Jul 18 03:06:05.978438 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.978417 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp"] Jul 18 03:06:05.981963 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:05.981941 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-43e0ec51-openshift-default-864bdf4c59-cdrqp"] Jul 18 03:06:06.757165 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:06.757132 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" path="/var/lib/kubelet/pods/8b4c62f6-c833-4724-b024-387d5b1cf5dd/volumes" Jul 18 03:06:17.938307 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:17.938268 2560 patch_prober.go:28] interesting pod/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 18 03:06:17.938805 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:17.938332 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 18 03:06:18.024667 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:18.024632 2560 generic.go:356] "Generic (PLEG): container finished" podID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerID="35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929" exitCode=0 Jul 18 03:06:18.024808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:18.024702 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" event={"ID":"57a6680a-817b-4c46-97fc-fc748dbe226e","Type":"ContainerDied","Data":"35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929"} Jul 18 03:06:18.025092 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:18.025078 2560 scope.go:117] "RemoveContainer" containerID="35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929" Jul 18 03:06:18.026170 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:18.026139 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:18.026249 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:18.026208 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:18.026288 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:18.026246 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:19.032869 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:19.032829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" event={"ID":"57a6680a-817b-4c46-97fc-fc748dbe226e","Type":"ContainerStarted","Data":"a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f"} Jul 18 03:06:19.172658 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:19.172627 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:19.172833 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:19.172784 2560 patch_prober.go:28] interesting pod/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 18 03:06:19.172878 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:19.172826 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 18 03:06:20.172333 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:20.172297 2560 patch_prober.go:28] interesting pod/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 18 03:06:20.172718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:20.172347 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 18 03:06:21.172274 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:21.172240 2560 patch_prober.go:28] interesting pod/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 18 03:06:21.172457 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:21.172291 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 18 03:06:22.172374 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:22.172330 2560 patch_prober.go:28] interesting pod/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 18 03:06:22.172545 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:22.172392 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 18 03:06:23.176360 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:23.176326 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:23.176806 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:23.176789 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:23.177389 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:23.177372 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:43.620613 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:43.620583 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v"] Jul 18 03:06:43.621194 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:43.620897 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" containerID="cri-o://a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f" gracePeriod=30 Jul 18 03:06:44.421255 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.421224 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx"] Jul 18 03:06:44.421619 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.421606 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" Jul 18 03:06:44.421664 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.421621 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" Jul 18 03:06:44.421708 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.421699 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" Jul 18 03:06:44.424783 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.424766 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.427374 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.427350 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-99930a7a-openshift-default-dockercfg-rcrwg\"" Jul 18 03:06:44.436599 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.436576 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx"] Jul 18 03:06:44.484622 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-envoy\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.484756 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-certs\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.484756 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-credential-socket\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.484756 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484732 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljnk5\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-kube-api-access-ljnk5\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.484932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-data\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.484932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484888 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-token\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.484932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-socket\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.485111 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.484956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istiod-ca-cert\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.485111 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.485019 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-podinfo\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.585649 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-data\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.585850 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-token\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.585850 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-socket\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.585850 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istiod-ca-cert\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.585850 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585741 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-podinfo\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.585850 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-envoy\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586167 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585868 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-certs\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586167 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-credential-socket\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586167 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.585935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljnk5\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-kube-api-access-ljnk5\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586167 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.586080 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-data\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586167 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.586113 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-socket\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586396 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.586241 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-certs\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586396 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.586314 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-credential-socket\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.586508 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.586489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istiod-ca-cert\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.588256 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.588231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-envoy\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.588483 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.588462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-podinfo\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.593227 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.593208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljnk5\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-kube-api-access-ljnk5\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.593348 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.593330 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-token\") pod \"e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.737106 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.737038 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:44.873173 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.873150 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx"] Jul 18 03:06:44.874051 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:06:44.874030 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a14b53c_2f2e_42e1_a66e_7e1daaadbd23.slice/crio-b8ce35967ea2ecb20bc459b07389c60b168b1d787aafa7f31b72774d1275fbb1 WatchSource:0}: Error finding container b8ce35967ea2ecb20bc459b07389c60b168b1d787aafa7f31b72774d1275fbb1: Status 404 returned error can't find the container with id b8ce35967ea2ecb20bc459b07389c60b168b1d787aafa7f31b72774d1275fbb1 Jul 18 03:06:44.876729 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.876699 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:44.876787 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.876760 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:44.876835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:44.876790 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:06:45.190096 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:45.190062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" event={"ID":"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23","Type":"ContainerStarted","Data":"6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4"} Jul 18 03:06:45.190301 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:45.190117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" event={"ID":"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23","Type":"ContainerStarted","Data":"b8ce35967ea2ecb20bc459b07389c60b168b1d787aafa7f31b72774d1275fbb1"} Jul 18 03:06:45.212732 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:45.212688 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podStartSLOduration=1.212672264 podStartE2EDuration="1.212672264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:06:45.21010311 +0000 UTC m=+1596.998876881" watchObservedRunningTime="2026-07-18 03:06:45.212672264 +0000 UTC m=+1597.001446034" Jul 18 03:06:45.738042 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:45.738008 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:45.742894 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:45.742869 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:46.204652 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:46.204612 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:46.205721 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:46.205625 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:06:48.877552 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:48.877528 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:49.033060 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033033 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-socket\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033060 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033063 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-envoy\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033109 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k72s5\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-kube-api-access-k72s5\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033128 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-token\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033156 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-podinfo\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033172 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-data\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033219 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-certs\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033509 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033298 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-credential-socket\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033509 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033336 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a6680a-817b-4c46-97fc-fc748dbe226e-istiod-ca-cert\") pod \"57a6680a-817b-4c46-97fc-fc748dbe226e\" (UID: \"57a6680a-817b-4c46-97fc-fc748dbe226e\") " Jul 18 03:06:49.033887 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033296 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:49.033887 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033803 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:49.033887 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033868 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:49.034147 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.033998 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a6680a-817b-4c46-97fc-fc748dbe226e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:06:49.034147 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.034070 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-data" (OuterVolumeSpecName: "istio-data") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:49.035539 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.035516 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-kube-api-access-k72s5" (OuterVolumeSpecName: "kube-api-access-k72s5") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "kube-api-access-k72s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:06:49.035631 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.035536 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:06:49.035781 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.035762 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:06:49.035842 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.035772 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-token" (OuterVolumeSpecName: "istio-token") pod "57a6680a-817b-4c46-97fc-fc748dbe226e" (UID: "57a6680a-817b-4c46-97fc-fc748dbe226e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:06:49.134546 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134518 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134546 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134539 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134546 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134549 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134557 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134566 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a6680a-817b-4c46-97fc-fc748dbe226e-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134573 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134581 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134588 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k72s5\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-kube-api-access-k72s5\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.134742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.134596 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a6680a-817b-4c46-97fc-fc748dbe226e-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:06:49.221333 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.221301 2560 generic.go:356] "Generic (PLEG): container finished" podID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerID="a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f" exitCode=0 Jul 18 03:06:49.221492 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.221351 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" Jul 18 03:06:49.221492 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.221393 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" event={"ID":"57a6680a-817b-4c46-97fc-fc748dbe226e","Type":"ContainerDied","Data":"a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f"} Jul 18 03:06:49.221492 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.221451 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v" event={"ID":"57a6680a-817b-4c46-97fc-fc748dbe226e","Type":"ContainerDied","Data":"1b64c9c0f0d29ced4aadde9e798636aa0c04ac471908657166de60c36e0de25b"} Jul 18 03:06:49.221492 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.221480 2560 scope.go:117] "RemoveContainer" containerID="a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f" Jul 18 03:06:49.231188 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.231169 2560 scope.go:117] "RemoveContainer" containerID="35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929" Jul 18 03:06:49.238944 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.238925 2560 scope.go:117] "RemoveContainer" containerID="a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f" Jul 18 03:06:49.239202 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:06:49.239183 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f\": container with ID starting with a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f not found: ID does not exist" containerID="a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f" Jul 18 03:06:49.239249 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.239210 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f"} err="failed to get container status \"a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f\": rpc error: code = NotFound desc = could not find container \"a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f\": container with ID starting with a8815bfbda564799b37b488b9d3774d5f85450e62f4d4a6d8180d3475798c98f not found: ID does not exist" Jul 18 03:06:49.239249 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.239227 2560 scope.go:117] "RemoveContainer" containerID="35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929" Jul 18 03:06:49.239463 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:06:49.239443 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929\": container with ID starting with 35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929 not found: ID does not exist" containerID="35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929" Jul 18 03:06:49.239528 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.239472 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929"} err="failed to get container status \"35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929\": rpc error: code = NotFound desc = could not find container \"35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929\": container with ID starting with 35469a537e6b29f1b9bab6fda3e538061e94635cb5cb1d0f5711e2564a3db929 not found: ID does not exist" Jul 18 03:06:49.243257 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.243221 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v"] Jul 18 03:06:49.247063 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:49.247043 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a70f0b50-gw-openshift-default-966cdfbb7-gxv8v"] Jul 18 03:06:50.758030 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:06:50.757990 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" path="/var/lib/kubelet/pods/57a6680a-817b-4c46-97fc-fc748dbe226e/volumes" Jul 18 03:07:01.206218 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.206183 2560 patch_prober.go:28] interesting pod/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 18 03:07:01.206631 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.206234 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 18 03:07:01.295037 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.295007 2560 generic.go:356] "Generic (PLEG): container finished" podID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerID="6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4" exitCode=0 Jul 18 03:07:01.295172 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.295086 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" event={"ID":"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23","Type":"ContainerDied","Data":"6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4"} Jul 18 03:07:01.295459 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.295443 2560 scope.go:117] "RemoveContainer" containerID="6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4" Jul 18 03:07:01.296419 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.296388 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:01.296517 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.296471 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:01.296580 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:01.296537 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:02.302510 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:02.302475 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" event={"ID":"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23","Type":"ContainerStarted","Data":"45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e"} Jul 18 03:07:02.737802 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:02.737703 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:07:02.738088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:02.737983 2560 patch_prober.go:28] interesting pod/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 18 03:07:02.738280 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:02.738256 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 18 03:07:03.737664 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:03.737626 2560 patch_prober.go:28] interesting pod/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 18 03:07:03.738255 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:03.738223 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 18 03:07:04.737770 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:04.737734 2560 patch_prober.go:28] interesting pod/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 18 03:07:04.738788 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:04.738369 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 18 03:07:05.737700 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:05.737658 2560 patch_prober.go:28] interesting pod/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 18 03:07:05.738016 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:05.737959 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 18 03:07:06.741895 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:06.741869 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:07:06.742331 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:06.742164 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:07:06.742771 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:06.742754 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:07:28.661109 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:28.661072 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx"] Jul 18 03:07:28.661644 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:28.661323 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" containerID="cri-o://45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e" gracePeriod=30 Jul 18 03:07:29.419196 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419158 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t"] Jul 18 03:07:29.419615 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419600 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" Jul 18 03:07:29.419615 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419616 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" Jul 18 03:07:29.419742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419626 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" Jul 18 03:07:29.419742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419632 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" Jul 18 03:07:29.419742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419703 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" Jul 18 03:07:29.419742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419712 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" Jul 18 03:07:29.419742 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.419720 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4c62f6-c833-4724-b024-387d5b1cf5dd" containerName="istio-proxy" Jul 18 03:07:29.422855 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.422835 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.425981 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.425952 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-efca33f7-openshift-default-dockercfg-6f7z2\"" Jul 18 03:07:29.434560 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.434538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t"] Jul 18 03:07:29.503445 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-podinfo\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503568 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf70c63-beb4-4130-a36c-f28e842bec4a-istiod-ca-cert\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503568 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-envoy\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503643 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-token\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503678 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503648 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcw6g\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-kube-api-access-zcw6g\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-certs\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-socket\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503720 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-credential-socket\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.503784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.503771 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-data\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605213 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-envoy\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-token\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcw6g\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-kube-api-access-zcw6g\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-certs\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-socket\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-credential-socket\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-data\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-podinfo\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605453 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf70c63-beb4-4130-a36c-f28e842bec4a-istiod-ca-cert\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605893 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605743 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-certs\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605893 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605804 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-data\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.605893 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605865 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-socket\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.606093 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.605935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-credential-socket\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.606146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.606087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf70c63-beb4-4130-a36c-f28e842bec4a-istiod-ca-cert\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.607920 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.607888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-envoy\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.608044 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.608026 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-podinfo\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.612829 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.612805 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-token\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.613071 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.613054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcw6g\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-kube-api-access-zcw6g\") pod \"e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.734089 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.734026 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:29.907595 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:07:29.907566 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf70c63_beb4_4130_a36c_f28e842bec4a.slice/crio-fe1fe4181148f627499d7e5ca4eccfdd5cf41570ad4b75d7d6243ab241076ff8 WatchSource:0}: Error finding container fe1fe4181148f627499d7e5ca4eccfdd5cf41570ad4b75d7d6243ab241076ff8: Status 404 returned error can't find the container with id fe1fe4181148f627499d7e5ca4eccfdd5cf41570ad4b75d7d6243ab241076ff8 Jul 18 03:07:29.907796 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.907771 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t"] Jul 18 03:07:29.909873 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.909843 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:29.909942 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.909907 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:29.910001 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:29.909938 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:30.460261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:30.460226 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" event={"ID":"bcf70c63-beb4-4130-a36c-f28e842bec4a","Type":"ContainerStarted","Data":"660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf"} Jul 18 03:07:30.460444 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:30.460282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" event={"ID":"bcf70c63-beb4-4130-a36c-f28e842bec4a","Type":"ContainerStarted","Data":"fe1fe4181148f627499d7e5ca4eccfdd5cf41570ad4b75d7d6243ab241076ff8"} Jul 18 03:07:30.480872 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:30.480825 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" podStartSLOduration=1.480810323 podStartE2EDuration="1.480810323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:07:30.478520095 +0000 UTC m=+1642.267293868" watchObservedRunningTime="2026-07-18 03:07:30.480810323 +0000 UTC m=+1642.269584093" Jul 18 03:07:30.734666 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:30.734593 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:30.739652 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:30.739625 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:31.466559 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:31.466522 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:31.467556 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:31.467536 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:33.912309 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:33.912285 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:07:34.049718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049691 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-token\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.049923 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049726 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljnk5\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-kube-api-access-ljnk5\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.049923 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049745 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-certs\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.049923 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049786 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-podinfo\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.049923 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049841 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-credential-socket\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.049923 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049906 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-data\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.050215 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.049954 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-socket\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.050215 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050018 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istiod-ca-cert\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.050215 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050046 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:07:34.050215 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050071 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-envoy\") pod \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\" (UID: \"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23\") " Jul 18 03:07:34.050215 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050183 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:07:34.050477 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050335 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:07:34.050531 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050518 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.050585 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050537 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.050585 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050553 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.050954 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050920 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:07:34.051116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.050958 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:07:34.051931 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.051901 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-kube-api-access-ljnk5" (OuterVolumeSpecName: "kube-api-access-ljnk5") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "kube-api-access-ljnk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:07:34.052044 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.052000 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:07:34.052253 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.052235 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:07:34.052306 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.052282 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" (UID: "6a14b53c-2f2e-42e1-a66e-7e1daaadbd23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:07:34.151052 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.151027 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.151052 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.151049 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.151201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.151058 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.151201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.151067 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.151201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.151074 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.151201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.151082 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljnk5\" (UniqueName: \"kubernetes.io/projected/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23-kube-api-access-ljnk5\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:07:34.485015 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.484910 2560 generic.go:356] "Generic (PLEG): container finished" podID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerID="45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e" exitCode=0 Jul 18 03:07:34.485015 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.484960 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" Jul 18 03:07:34.485015 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.484997 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" event={"ID":"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23","Type":"ContainerDied","Data":"45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e"} Jul 18 03:07:34.485244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.485059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx" event={"ID":"6a14b53c-2f2e-42e1-a66e-7e1daaadbd23","Type":"ContainerDied","Data":"b8ce35967ea2ecb20bc459b07389c60b168b1d787aafa7f31b72774d1275fbb1"} Jul 18 03:07:34.485244 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.485089 2560 scope.go:117] "RemoveContainer" containerID="45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e" Jul 18 03:07:34.495242 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.495225 2560 scope.go:117] "RemoveContainer" containerID="6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4" Jul 18 03:07:34.505284 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.505233 2560 scope.go:117] "RemoveContainer" containerID="45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e" Jul 18 03:07:34.505579 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:07:34.505557 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e\": container with ID starting with 45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e not found: ID does not exist" containerID="45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e" Jul 18 03:07:34.505666 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.505587 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e"} err="failed to get container status \"45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e\": rpc error: code = NotFound desc = could not find container \"45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e\": container with ID starting with 45da90fe62d35416efc9366d2cb3705646820dc4f7bcdcda2db19e4b6846e89e not found: ID does not exist" Jul 18 03:07:34.505666 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.505604 2560 scope.go:117] "RemoveContainer" containerID="6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4" Jul 18 03:07:34.505848 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:07:34.505828 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4\": container with ID starting with 6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4 not found: ID does not exist" containerID="6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4" Jul 18 03:07:34.505907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.505854 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4"} err="failed to get container status \"6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4\": rpc error: code = NotFound desc = could not find container \"6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4\": container with ID starting with 6ce62e7d95fbe6a912406b1a5a58694570befe244910c6b5bbe4ee31dfd01bb4 not found: ID does not exist" Jul 18 03:07:34.508350 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.508287 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx"] Jul 18 03:07:34.511237 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.511214 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-99930a7a-openshift-default-bf7f9df57-2jccx"] Jul 18 03:07:34.757433 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:34.757358 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" path="/var/lib/kubelet/pods/6a14b53c-2f2e-42e1-a66e-7e1daaadbd23/volumes" Jul 18 03:07:45.546875 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:45.546838 2560 generic.go:356] "Generic (PLEG): container finished" podID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerID="660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf" exitCode=0 Jul 18 03:07:45.547355 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:45.546908 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" event={"ID":"bcf70c63-beb4-4130-a36c-f28e842bec4a","Type":"ContainerDied","Data":"660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf"} Jul 18 03:07:45.547355 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:45.547282 2560 scope.go:117] "RemoveContainer" containerID="660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf" Jul 18 03:07:45.548249 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:45.548220 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:45.548351 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:45.548285 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:45.548351 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:45.548316 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:07:46.466443 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:46.466397 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:46.559293 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:46.559250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" event={"ID":"bcf70c63-beb4-4130-a36c-f28e842bec4a","Type":"ContainerStarted","Data":"43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc"} Jul 18 03:07:46.734633 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:46.734535 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:46.735529 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:46.735496 2560 patch_prober.go:28] interesting pod/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 18 03:07:46.735664 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:46.735582 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 18 03:07:47.735043 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:47.735003 2560 patch_prober.go:28] interesting pod/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 18 03:07:47.735533 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:47.735069 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 18 03:07:48.735411 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:48.735374 2560 patch_prober.go:28] interesting pod/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 18 03:07:48.736064 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:48.736030 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 18 03:07:49.735407 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:49.735366 2560 patch_prober.go:28] interesting pod/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 18 03:07:49.735778 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:49.735743 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 18 03:07:50.738007 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:50.737959 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:50.738452 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:50.738293 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:07:50.738921 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:07:50.738904 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:08:11.925236 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:11.925204 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t"] Jul 18 03:08:11.925698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:11.925419 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" containerID="cri-o://43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc" gracePeriod=30 Jul 18 03:08:17.182081 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.182051 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:08:17.340941 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.340912 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-credential-socket\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341102 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341040 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-data\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341102 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341076 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcw6g\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-kube-api-access-zcw6g\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341208 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341108 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf70c63-beb4-4130-a36c-f28e842bec4a-istiod-ca-cert\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341208 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341140 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-podinfo\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341208 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341172 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-socket\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341208 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341195 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-certs\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341412 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341168 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:08:17.341412 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341265 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-token\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341412 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341306 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-envoy\") pod \"bcf70c63-beb4-4130-a36c-f28e842bec4a\" (UID: \"bcf70c63-beb4-4130-a36c-f28e842bec4a\") " Jul 18 03:08:17.341569 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341437 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:08:17.341628 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341580 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:08:17.341628 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341591 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf70c63-beb4-4130-a36c-f28e842bec4a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:08:17.341744 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341700 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.341744 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341713 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf70c63-beb4-4130-a36c-f28e842bec4a-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.341744 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341722 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.341744 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.341731 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.342086 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.342063 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-data" (OuterVolumeSpecName: "istio-data") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:08:17.343552 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.343529 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:08:17.343655 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.343532 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:08:17.343752 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.343735 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-kube-api-access-zcw6g" (OuterVolumeSpecName: "kube-api-access-zcw6g") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "kube-api-access-zcw6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:08:17.343810 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.343746 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bcf70c63-beb4-4130-a36c-f28e842bec4a" (UID: "bcf70c63-beb4-4130-a36c-f28e842bec4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:08:17.442306 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.442285 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.442417 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.442309 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcw6g\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-kube-api-access-zcw6g\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.442417 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.442323 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.442417 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.442335 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.442417 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.442348 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf70c63-beb4-4130-a36c-f28e842bec4a-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:08:17.742475 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.742404 2560 generic.go:356] "Generic (PLEG): container finished" podID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerID="43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc" exitCode=0 Jul 18 03:08:17.742475 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.742463 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" Jul 18 03:08:17.742621 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.742478 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" event={"ID":"bcf70c63-beb4-4130-a36c-f28e842bec4a","Type":"ContainerDied","Data":"43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc"} Jul 18 03:08:17.744576 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.743018 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t" event={"ID":"bcf70c63-beb4-4130-a36c-f28e842bec4a","Type":"ContainerDied","Data":"fe1fe4181148f627499d7e5ca4eccfdd5cf41570ad4b75d7d6243ab241076ff8"} Jul 18 03:08:17.744576 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.743081 2560 scope.go:117] "RemoveContainer" containerID="43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc" Jul 18 03:08:17.759307 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.759289 2560 scope.go:117] "RemoveContainer" containerID="660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf" Jul 18 03:08:17.767748 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.767729 2560 scope.go:117] "RemoveContainer" containerID="43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc" Jul 18 03:08:17.768044 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:08:17.768023 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc\": container with ID starting with 43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc not found: ID does not exist" containerID="43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc" Jul 18 03:08:17.768127 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.768056 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc"} err="failed to get container status \"43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc\": rpc error: code = NotFound desc = could not find container \"43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc\": container with ID starting with 43fdc43acbfd67c7afc38dbd2b2ab24ca5143a793f28ea9c7a9dabe5e89fe7cc not found: ID does not exist" Jul 18 03:08:17.768127 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.768078 2560 scope.go:117] "RemoveContainer" containerID="660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf" Jul 18 03:08:17.768528 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:08:17.768409 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf\": container with ID starting with 660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf not found: ID does not exist" containerID="660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf" Jul 18 03:08:17.768528 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.768443 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf"} err="failed to get container status \"660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf\": rpc error: code = NotFound desc = could not find container \"660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf\": container with ID starting with 660718335aeb4a9c8fbd478c4860290bfd09b3032bad244ccb1f030604cb1faf not found: ID does not exist" Jul 18 03:08:17.770060 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.770040 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t"] Jul 18 03:08:17.772251 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:17.772233 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-efca33f7-openshift-default-645d76b6b7-xlm5t"] Jul 18 03:08:18.756314 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:08:18.756283 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" path="/var/lib/kubelet/pods/bcf70c63-beb4-4130-a36c-f28e842bec4a/volumes" Jul 18 03:09:54.505541 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505504 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk"] Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505895 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505906 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505914 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505919 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505929 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505935 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a6680a-817b-4c46-97fc-fc748dbe226e" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505941 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505946 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505954 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.505959 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.506029 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" Jul 18 03:09:54.506146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.506039 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" Jul 18 03:09:54.508220 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.508202 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.510866 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.510841 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1d958dfb-openshift-default-dockercfg-xlqhk\"" Jul 18 03:09:54.526820 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.526797 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk"] Jul 18 03:09:54.558496 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-credential-socket\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558610 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-envoy\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558610 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558527 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5h8\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-kube-api-access-tw5h8\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558610 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istiod-ca-cert\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558610 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-socket\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558762 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-certs\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558762 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-data\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558762 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-token\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.558762 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.558728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-podinfo\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659315 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-data\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659315 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-token\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659536 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659336 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-podinfo\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659536 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-credential-socket\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659536 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659396 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-envoy\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659711 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5h8\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-kube-api-access-tw5h8\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659770 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659726 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-data\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659770 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659735 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-credential-socket\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.659902 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istiod-ca-cert\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.660006 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.659988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-socket\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.660087 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.660030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-certs\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.660372 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.660349 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-socket\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.660462 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.660405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-certs\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.660626 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.660602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istiod-ca-cert\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.661852 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.661825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-envoy\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.662127 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.662106 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-podinfo\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.667984 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.667948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-token\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.668566 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.668541 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5h8\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-kube-api-access-tw5h8\") pod \"e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.819005 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.818956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:54.952784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.952750 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk"] Jul 18 03:09:54.954469 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:09:54.954438 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b4dd61_e76e_4e11_aa6a_1ebc03e29656.slice/crio-31caa53352576288ae48875977405ea0b69895c9c4d4a7f1d3f8cf382cdcd897 WatchSource:0}: Error finding container 31caa53352576288ae48875977405ea0b69895c9c4d4a7f1d3f8cf382cdcd897: Status 404 returned error can't find the container with id 31caa53352576288ae48875977405ea0b69895c9c4d4a7f1d3f8cf382cdcd897 Jul 18 03:09:54.957264 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.957208 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:09:54.957360 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.957306 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:09:54.957360 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:54.957347 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:09:55.267856 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:55.267821 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" event={"ID":"29b4dd61-e76e-4e11-aa6a-1ebc03e29656","Type":"ContainerStarted","Data":"e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3"} Jul 18 03:09:55.268042 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:55.267875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" event={"ID":"29b4dd61-e76e-4e11-aa6a-1ebc03e29656","Type":"ContainerStarted","Data":"31caa53352576288ae48875977405ea0b69895c9c4d4a7f1d3f8cf382cdcd897"} Jul 18 03:09:55.288324 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:55.288276 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" podStartSLOduration=1.288258831 podStartE2EDuration="1.288258831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:09:55.287643166 +0000 UTC m=+1787.076416937" watchObservedRunningTime="2026-07-18 03:09:55.288258831 +0000 UTC m=+1787.077032602" Jul 18 03:09:55.819557 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:55.819520 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:56.823803 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:56.823775 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:57.279287 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:57.279254 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:09:57.280162 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:09:57.280144 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:08.341569 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.341501 2560 generic.go:356] "Generic (PLEG): container finished" podID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerID="e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3" exitCode=0 Jul 18 03:10:08.341569 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.341561 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" event={"ID":"29b4dd61-e76e-4e11-aa6a-1ebc03e29656","Type":"ContainerDied","Data":"e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3"} Jul 18 03:10:08.342004 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.341903 2560 scope.go:117] "RemoveContainer" containerID="e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3" Jul 18 03:10:08.342827 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.342797 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:10:08.342880 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.342866 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:10:08.342925 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.342895 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:10:08.818010 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.817984 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:10:08.826468 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:08.826448 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:10:09.353951 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:09.353905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" event={"ID":"29b4dd61-e76e-4e11-aa6a-1ebc03e29656","Type":"ContainerStarted","Data":"285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a"} Jul 18 03:10:09.819610 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:09.819580 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:09.820801 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:09.820776 2560 patch_prober.go:28] interesting pod/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 18 03:10:09.820929 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:09.820823 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 18 03:10:10.819869 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:10.819837 2560 patch_prober.go:28] interesting pod/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 18 03:10:10.820332 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:10.819899 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 18 03:10:11.820020 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:11.819960 2560 patch_prober.go:28] interesting pod/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 18 03:10:11.820404 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:11.820040 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 18 03:10:12.279143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:12.279114 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:12.819856 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:12.819823 2560 patch_prober.go:28] interesting pod/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 18 03:10:12.820058 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:12.819896 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 18 03:10:13.823584 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:13.823553 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:13.824532 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:13.824508 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:30.413393 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:30.413360 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk"] Jul 18 03:10:30.413873 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:30.413616 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" containerID="cri-o://285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a" gracePeriod=30 Jul 18 03:10:35.659071 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.659048 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:35.745094 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745025 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-socket\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745094 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745060 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-certs\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745291 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-data\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745291 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745156 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw5h8\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-kube-api-access-tw5h8\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745291 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745190 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istiod-ca-cert\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745291 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745242 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-token\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745505 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745288 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-envoy\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745505 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745304 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:10:35.745505 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745323 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-credential-socket\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745505 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745333 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:10:35.745505 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745456 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-podinfo\") pod \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\" (UID: \"29b4dd61-e76e-4e11-aa6a-1ebc03e29656\") " Jul 18 03:10:35.745777 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745498 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:10:35.745777 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745569 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:10:35.745888 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745835 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.745888 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745856 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.745888 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745873 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.745888 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.745885 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.746164 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.746006 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-data" (OuterVolumeSpecName: "istio-data") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:10:35.747556 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.747530 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-token" (OuterVolumeSpecName: "istio-token") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:10:35.747657 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.747535 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:10:35.747657 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.747566 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-kube-api-access-tw5h8" (OuterVolumeSpecName: "kube-api-access-tw5h8") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "kube-api-access-tw5h8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:10:35.747657 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.747627 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29b4dd61-e76e-4e11-aa6a-1ebc03e29656" (UID: "29b4dd61-e76e-4e11-aa6a-1ebc03e29656"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:10:35.847413 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.847383 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.847413 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.847408 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw5h8\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-kube-api-access-tw5h8\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.847571 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.847419 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.847571 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.847428 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:35.847571 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:35.847436 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b4dd61-e76e-4e11-aa6a-1ebc03e29656-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:10:36.500730 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.500695 2560 generic.go:356] "Generic (PLEG): container finished" podID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerID="285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a" exitCode=0 Jul 18 03:10:36.500907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.500749 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" Jul 18 03:10:36.500907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.500781 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" event={"ID":"29b4dd61-e76e-4e11-aa6a-1ebc03e29656","Type":"ContainerDied","Data":"285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a"} Jul 18 03:10:36.500907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.500839 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk" event={"ID":"29b4dd61-e76e-4e11-aa6a-1ebc03e29656","Type":"ContainerDied","Data":"31caa53352576288ae48875977405ea0b69895c9c4d4a7f1d3f8cf382cdcd897"} Jul 18 03:10:36.500907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.500862 2560 scope.go:117] "RemoveContainer" containerID="285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a" Jul 18 03:10:36.510635 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.510616 2560 scope.go:117] "RemoveContainer" containerID="e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3" Jul 18 03:10:36.518804 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.518786 2560 scope.go:117] "RemoveContainer" containerID="285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a" Jul 18 03:10:36.519090 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:10:36.519071 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a\": container with ID starting with 285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a not found: ID does not exist" containerID="285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a" Jul 18 03:10:36.519145 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.519098 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a"} err="failed to get container status \"285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a\": rpc error: code = NotFound desc = could not find container \"285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a\": container with ID starting with 285580876b3a57687a2485f5011e9daf0d4e3ed731049ba030486237912a128a not found: ID does not exist" Jul 18 03:10:36.519145 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.519115 2560 scope.go:117] "RemoveContainer" containerID="e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3" Jul 18 03:10:36.519350 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:10:36.519329 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3\": container with ID starting with e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3 not found: ID does not exist" containerID="e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3" Jul 18 03:10:36.519410 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.519359 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3"} err="failed to get container status \"e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3\": rpc error: code = NotFound desc = could not find container \"e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3\": container with ID starting with e9107b98ac75fee23346a9faa35b213be966c440cb51c572257cb3d9df643ac3 not found: ID does not exist" Jul 18 03:10:36.523158 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.523136 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk"] Jul 18 03:10:36.527212 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.527184 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1d958dfb-openshift-default-77dc75567f-fw7pk"] Jul 18 03:10:36.757253 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:36.757170 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" path="/var/lib/kubelet/pods/29b4dd61-e76e-4e11-aa6a-1ebc03e29656/volumes" Jul 18 03:10:49.876452 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.876422 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h"] Jul 18 03:10:49.876962 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.876943 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" Jul 18 03:10:49.877081 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.876980 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" Jul 18 03:10:49.877081 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.876995 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" Jul 18 03:10:49.877081 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.877004 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" Jul 18 03:10:49.877234 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.877121 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" Jul 18 03:10:49.877234 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.877135 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf70c63-beb4-4130-a36c-f28e842bec4a" containerName="istio-proxy" Jul 18 03:10:49.877234 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.877145 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b4dd61-e76e-4e11-aa6a-1ebc03e29656" containerName="istio-proxy" Jul 18 03:10:49.877234 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.877156 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a14b53c-2f2e-42e1-a66e-7e1daaadbd23" containerName="istio-proxy" Jul 18 03:10:49.879551 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.879530 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.882019 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.881950 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7aa900e3-openshift-default-dockercfg-f8hfg\"" Jul 18 03:10:49.891614 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.891592 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h"] Jul 18 03:10:49.962925 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.962897 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-socket\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.962935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-credential-socket\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.962963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istiod-ca-cert\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.963003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-certs\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.963026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-token\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.963075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-envoy\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.963095 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6bc5\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-kube-api-access-j6bc5\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963138 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.963131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-podinfo\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:49.963407 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:49.963182 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-data\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064132 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-socket\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-credential-socket\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istiod-ca-cert\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064192 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-certs\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-token\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-envoy\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064281 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6bc5\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-kube-api-access-j6bc5\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-podinfo\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064689 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-data\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064689 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064539 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-socket\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064869 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064745 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-certs\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.064869 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.064790 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-credential-socket\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.065056 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.065034 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-data\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.065189 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.065164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istiod-ca-cert\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.066859 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.066841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-envoy\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.067099 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.067079 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-podinfo\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.072358 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.072335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-token\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.072486 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.072466 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6bc5\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-kube-api-access-j6bc5\") pod \"e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.191681 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.191621 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:50.325468 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.325440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h"] Jul 18 03:10:50.326109 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:10:50.326073 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5eae5e3_30e9_4349_9466_e0a567ce69c0.slice/crio-250b71384da864f1e991dbf3acd216701ca05fa7a6f2fa31d28401c0f34c8130 WatchSource:0}: Error finding container 250b71384da864f1e991dbf3acd216701ca05fa7a6f2fa31d28401c0f34c8130: Status 404 returned error can't find the container with id 250b71384da864f1e991dbf3acd216701ca05fa7a6f2fa31d28401c0f34c8130 Jul 18 03:10:50.328200 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.328170 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:10:50.328277 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.328231 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:10:50.328277 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.328261 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:10:50.587796 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.587754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" event={"ID":"f5eae5e3-30e9-4349-9466-e0a567ce69c0","Type":"ContainerStarted","Data":"c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc"} Jul 18 03:10:50.588023 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.587806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" event={"ID":"f5eae5e3-30e9-4349-9466-e0a567ce69c0","Type":"ContainerStarted","Data":"250b71384da864f1e991dbf3acd216701ca05fa7a6f2fa31d28401c0f34c8130"} Jul 18 03:10:50.609151 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:50.609093 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podStartSLOduration=1.609072678 podStartE2EDuration="1.609072678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:10:50.608027133 +0000 UTC m=+1842.396800907" watchObservedRunningTime="2026-07-18 03:10:50.609072678 +0000 UTC m=+1842.397846449" Jul 18 03:10:51.192126 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:51.192092 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:51.196571 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:51.196547 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:51.593122 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:51.593092 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:10:51.594091 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:10:51.594074 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:11:06.594697 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.594641 2560 patch_prober.go:28] interesting pod/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 18 03:11:06.595056 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.594708 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 18 03:11:06.687154 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.687093 2560 generic.go:356] "Generic (PLEG): container finished" podID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerID="c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc" exitCode=0 Jul 18 03:11:06.687253 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.687158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" event={"ID":"f5eae5e3-30e9-4349-9466-e0a567ce69c0","Type":"ContainerDied","Data":"c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc"} Jul 18 03:11:06.687504 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.687491 2560 scope.go:117] "RemoveContainer" containerID="c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc" Jul 18 03:11:06.688485 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.688457 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:11:06.688542 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.688520 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:11:06.688578 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:06.688555 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:11:07.695205 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:07.695171 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" event={"ID":"f5eae5e3-30e9-4349-9466-e0a567ce69c0","Type":"ContainerStarted","Data":"988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d"} Jul 18 03:11:08.192646 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:08.192606 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:11:08.192840 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:08.192815 2560 patch_prober.go:28] interesting pod/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 18 03:11:08.192913 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:08.192869 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 18 03:11:09.192027 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:09.191923 2560 patch_prober.go:28] interesting pod/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 18 03:11:09.192027 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:09.192001 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 18 03:11:10.192814 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:10.192776 2560 patch_prober.go:28] interesting pod/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 18 03:11:10.193252 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:10.192843 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 18 03:11:11.191934 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:11.191895 2560 patch_prober.go:28] interesting pod/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 18 03:11:11.192143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:11.191952 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 18 03:11:12.196117 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:12.196085 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:11:12.196512 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:12.196482 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:11:12.197236 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:11:12.197216 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:12:45.999192 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:45.999160 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h"] Jul 18 03:12:45.999788 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:45.999493 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" containerID="cri-o://988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d" gracePeriod=30 Jul 18 03:12:51.247835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.247810 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:12:51.253722 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.253698 2560 generic.go:356] "Generic (PLEG): container finished" podID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerID="988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d" exitCode=0 Jul 18 03:12:51.253835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.253738 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" Jul 18 03:12:51.253835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.253772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" event={"ID":"f5eae5e3-30e9-4349-9466-e0a567ce69c0","Type":"ContainerDied","Data":"988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d"} Jul 18 03:12:51.253835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.253811 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h" event={"ID":"f5eae5e3-30e9-4349-9466-e0a567ce69c0","Type":"ContainerDied","Data":"250b71384da864f1e991dbf3acd216701ca05fa7a6f2fa31d28401c0f34c8130"} Jul 18 03:12:51.253835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.253833 2560 scope.go:117] "RemoveContainer" containerID="988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d" Jul 18 03:12:51.262346 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.262330 2560 scope.go:117] "RemoveContainer" containerID="c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc" Jul 18 03:12:51.271824 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.271803 2560 scope.go:117] "RemoveContainer" containerID="988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d" Jul 18 03:12:51.272147 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:12:51.272126 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d\": container with ID starting with 988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d not found: ID does not exist" containerID="988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d" Jul 18 03:12:51.272277 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.272159 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d"} err="failed to get container status \"988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d\": rpc error: code = NotFound desc = could not find container \"988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d\": container with ID starting with 988ae7f9d45e97014ec9c6d2a40e1ecc368cd2be0fac3212f7341aa6f336e60d not found: ID does not exist" Jul 18 03:12:51.272277 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.272181 2560 scope.go:117] "RemoveContainer" containerID="c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc" Jul 18 03:12:51.272571 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:12:51.272550 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc\": container with ID starting with c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc not found: ID does not exist" containerID="c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc" Jul 18 03:12:51.272647 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.272577 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc"} err="failed to get container status \"c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc\": rpc error: code = NotFound desc = could not find container \"c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc\": container with ID starting with c76cf672cf7328ca936d37e0b9e95c31540a891ad4d94bfbe092c57be5e71afc not found: ID does not exist" Jul 18 03:12:51.358557 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358526 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istiod-ca-cert\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.358557 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358562 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-certs\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.358809 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358584 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6bc5\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-kube-api-access-j6bc5\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.358809 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358773 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-data\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.358932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358831 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-token\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.358932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358843 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:12:51.358932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358874 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-envoy\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.358932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.358919 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-credential-socket\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.359181 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-socket\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.359181 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359036 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-podinfo\") pod \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\" (UID: \"f5eae5e3-30e9-4349-9466-e0a567ce69c0\") " Jul 18 03:12:51.359181 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359063 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:12:51.359344 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359211 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:12:51.359398 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359347 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:12:51.359460 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359442 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.359516 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359465 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.359516 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359479 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.359516 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359492 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.359665 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.359643 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:12:51.361406 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.361377 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-kube-api-access-j6bc5" (OuterVolumeSpecName: "kube-api-access-j6bc5") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "kube-api-access-j6bc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:12:51.361509 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.361418 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:12:51.361509 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.361429 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:12:51.361509 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.361443 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5eae5e3-30e9-4349-9466-e0a567ce69c0" (UID: "f5eae5e3-30e9-4349-9466-e0a567ce69c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:12:51.460377 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.460343 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.460377 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.460369 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6bc5\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-kube-api-access-j6bc5\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.460377 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.460380 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.460377 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.460388 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.460702 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.460396 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5eae5e3-30e9-4349-9466-e0a567ce69c0-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:12:51.578388 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.578360 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h"] Jul 18 03:12:51.581906 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:51.581879 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7aa900e3-openshift-default-67bddf7f-bzt4h"] Jul 18 03:12:52.757143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:12:52.757108 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" path="/var/lib/kubelet/pods/f5eae5e3-30e9-4349-9466-e0a567ce69c0/volumes" Jul 18 03:13:41.941016 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.940982 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s"] Jul 18 03:13:41.941479 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.941396 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" Jul 18 03:13:41.941479 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.941408 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" Jul 18 03:13:41.941479 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.941419 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" Jul 18 03:13:41.941479 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.941425 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" Jul 18 03:13:41.941613 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.941500 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" Jul 18 03:13:41.941613 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.941511 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5eae5e3-30e9-4349-9466-e0a567ce69c0" containerName="istio-proxy" Jul 18 03:13:41.944634 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.944615 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.947358 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.947338 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-69f74a-openshift-default-dockercfg-9k2tj\"" Jul 18 03:13:41.958150 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.958127 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s"] Jul 18 03:13:41.996027 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.995999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7c96\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-kube-api-access-m7c96\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-socket\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996064 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istiod-ca-cert\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996120 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-data\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996286 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-token\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996286 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996196 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-envoy\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996286 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-podinfo\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996286 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996254 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-certs\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:41.996286 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:41.996275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-credential-socket\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097301 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7c96\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-kube-api-access-m7c96\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097463 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-socket\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097463 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istiod-ca-cert\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097463 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097369 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-data\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097463 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-token\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097463 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-envoy\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097708 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-podinfo\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097708 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-certs\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.097708 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-credential-socket\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.098009 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.097906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-data\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.098009 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.098000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-credential-socket\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.098179 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.098098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-socket\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.098179 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.098154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-certs\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.098398 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.098380 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istiod-ca-cert\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.099745 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.099727 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-envoy\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.100213 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.100198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-podinfo\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.105160 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.105137 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-token\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.105300 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.105187 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7c96\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-kube-api-access-m7c96\") pod \"e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.255400 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.255308 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:42.399871 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.399841 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s"] Jul 18 03:13:42.401066 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:13:42.401042 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb64808_c7a9_4880_9a20_db1e53f7e1e2.slice/crio-585f516446771bea9113dba796739ca7d645bc7003d9f60bb551b03469179989 WatchSource:0}: Error finding container 585f516446771bea9113dba796739ca7d645bc7003d9f60bb551b03469179989: Status 404 returned error can't find the container with id 585f516446771bea9113dba796739ca7d645bc7003d9f60bb551b03469179989 Jul 18 03:13:42.403255 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.403225 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:13:42.403320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.403287 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:13:42.403320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.403317 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:13:42.540412 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.540376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" event={"ID":"ffb64808-c7a9-4880-9a20-db1e53f7e1e2","Type":"ContainerStarted","Data":"fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86"} Jul 18 03:13:42.540584 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.540441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" event={"ID":"ffb64808-c7a9-4880-9a20-db1e53f7e1e2","Type":"ContainerStarted","Data":"585f516446771bea9113dba796739ca7d645bc7003d9f60bb551b03469179989"} Jul 18 03:13:42.558391 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:42.558344 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podStartSLOduration=1.5583279810000001 podStartE2EDuration="1.558327981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:13:42.557661697 +0000 UTC m=+2014.346435467" watchObservedRunningTime="2026-07-18 03:13:42.558327981 +0000 UTC m=+2014.347101750" Jul 18 03:13:43.256119 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:43.256084 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:43.260662 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:43.260638 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:43.546207 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:43.546178 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:43.547023 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:43.547000 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:13:49.709222 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.709184 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72"] Jul 18 03:13:49.713487 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.713462 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.715950 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.715927 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-cde1c8-openshift-default-dockercfg-9tlgb\"" Jul 18 03:13:49.724792 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.724769 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72"] Jul 18 03:13:49.767834 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.767808 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-certs\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.767964 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.767847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxtsp\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-kube-api-access-cxtsp\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.767964 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.767870 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-socket\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.767964 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.767957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-podinfo\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.768116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.768055 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-envoy\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.768116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.768090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-data\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.768116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.768111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-token\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.768214 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.768147 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-credential-socket\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.768214 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.768168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istiod-ca-cert\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869004 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.868949 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-envoy\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869176 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-data\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869176 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869046 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-token\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869176 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-credential-socket\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869176 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istiod-ca-cert\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869176 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-certs\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869476 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxtsp\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-kube-api-access-cxtsp\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869476 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-socket\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869476 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-podinfo\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869633 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869488 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-data\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869720 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869692 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-certs\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.869783 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.869746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-credential-socket\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.870366 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.870342 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-socket\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.870664 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.870642 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istiod-ca-cert\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.871840 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.871815 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-envoy\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.871955 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.871854 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-podinfo\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.877444 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.877424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-token\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:49.878137 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:49.878113 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxtsp\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-kube-api-access-cxtsp\") pod \"e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:50.025274 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.025242 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:50.159834 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.159810 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72"] Jul 18 03:13:50.161075 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:13:50.161045 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ba8e0d_9c16_48ee_9ae7_afa8915a7878.slice/crio-cb9e2b461693c463902aba09e5102e94604733cae84997a8a486568f5edfd62b WatchSource:0}: Error finding container cb9e2b461693c463902aba09e5102e94604733cae84997a8a486568f5edfd62b: Status 404 returned error can't find the container with id cb9e2b461693c463902aba09e5102e94604733cae84997a8a486568f5edfd62b Jul 18 03:13:50.163216 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.163181 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:13:50.163320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.163249 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:13:50.163320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.163275 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:13:50.586889 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.586854 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" event={"ID":"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878","Type":"ContainerStarted","Data":"4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0"} Jul 18 03:13:50.587102 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.586911 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" event={"ID":"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878","Type":"ContainerStarted","Data":"cb9e2b461693c463902aba09e5102e94604733cae84997a8a486568f5edfd62b"} Jul 18 03:13:50.606166 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:50.606105 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podStartSLOduration=1.606088016 podStartE2EDuration="1.606088016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:13:50.605249864 +0000 UTC m=+2022.394023647" watchObservedRunningTime="2026-07-18 03:13:50.606088016 +0000 UTC m=+2022.394861875" Jul 18 03:13:51.026201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:51.026165 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:51.031685 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:51.031661 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:51.592154 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:51.592125 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:13:51.592998 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:13:51.592960 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:14:02.665680 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:02.665646 2560 generic.go:356] "Generic (PLEG): container finished" podID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerID="fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86" exitCode=0 Jul 18 03:14:02.666126 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:02.665690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" event={"ID":"ffb64808-c7a9-4880-9a20-db1e53f7e1e2","Type":"ContainerDied","Data":"fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86"} Jul 18 03:14:02.666126 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:02.666090 2560 scope.go:117] "RemoveContainer" containerID="fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86" Jul 18 03:14:02.667115 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:02.667081 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:14:02.667251 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:02.667150 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:14:02.667251 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:02.667181 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:14:03.680084 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:03.680035 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" event={"ID":"ffb64808-c7a9-4880-9a20-db1e53f7e1e2","Type":"ContainerStarted","Data":"84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12"} Jul 18 03:14:04.256177 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:04.256140 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:14:04.256564 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:04.256474 2560 patch_prober.go:28] interesting pod/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 18 03:14:04.256747 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:04.256724 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 18 03:14:05.256149 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:05.256084 2560 patch_prober.go:28] interesting pod/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 18 03:14:05.256660 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:05.256172 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 18 03:14:06.257222 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:06.256764 2560 patch_prober.go:28] interesting pod/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 18 03:14:06.257222 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:06.256831 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 18 03:14:07.256660 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:07.256623 2560 patch_prober.go:28] interesting pod/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 18 03:14:07.256957 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:07.256906 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 18 03:14:07.634142 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:07.634099 2560 patch_prober.go:28] interesting pod/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 03:14:07.634745 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:07.634695 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 03:14:08.259498 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.259468 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:14:08.259876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.259855 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:14:08.260533 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.260500 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:14:08.717634 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.717555 2560 generic.go:356] "Generic (PLEG): container finished" podID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerID="4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0" exitCode=0 Jul 18 03:14:08.718058 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.717627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" event={"ID":"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878","Type":"ContainerDied","Data":"4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0"} Jul 18 03:14:08.718143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.718128 2560 scope.go:117] "RemoveContainer" containerID="4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0" Jul 18 03:14:08.719074 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.719047 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:14:08.719187 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.719122 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:14:08.719187 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:08.719171 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:14:09.725876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:09.725843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" event={"ID":"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878","Type":"ContainerStarted","Data":"f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f"} Jul 18 03:14:10.026064 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:10.026037 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:14:10.026262 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:10.026239 2560 patch_prober.go:28] interesting pod/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 18 03:14:10.026318 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:10.026299 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 18 03:14:11.026007 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:11.025949 2560 patch_prober.go:28] interesting pod/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 18 03:14:11.026389 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:11.026022 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 18 03:14:12.025703 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:12.025668 2560 patch_prober.go:28] interesting pod/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 18 03:14:12.025911 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:12.025726 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 18 03:14:13.026474 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:13.026442 2560 patch_prober.go:28] interesting pod/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 18 03:14:13.026932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:13.026496 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 18 03:14:14.029332 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:14.029296 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:14:14.029811 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:14.029681 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:14:14.030363 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:14:14.030343 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:15:03.156304 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:03.156229 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s"] Jul 18 03:15:03.156813 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:03.156429 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" containerID="cri-o://84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12" gracePeriod=30 Jul 18 03:15:08.260987 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.260934 2560 patch_prober.go:28] interesting pod/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 18 03:15:08.261370 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.261018 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 18 03:15:08.404819 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.404787 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:15:08.500386 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500313 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-certs\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500386 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500362 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-podinfo\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500591 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500392 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-data\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500591 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500420 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-envoy\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500591 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500467 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-token\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500591 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500502 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7c96\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-kube-api-access-m7c96\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500591 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500580 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-credential-socket\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.500819 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500651 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:08.500883 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500846 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:08.500996 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.500951 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istiod-ca-cert\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.501123 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501049 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-socket\") pod \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\" (UID: \"ffb64808-c7a9-4880-9a20-db1e53f7e1e2\") " Jul 18 03:15:08.501316 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501294 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:08.501413 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501388 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:15:08.501465 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501426 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:08.501687 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501670 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.501737 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501694 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.501737 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501708 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.501737 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501722 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.501737 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.501735 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.502753 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.502731 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:15:08.502828 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.502752 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:15:08.503044 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.503023 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:08.503092 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.503051 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-kube-api-access-m7c96" (OuterVolumeSpecName: "kube-api-access-m7c96") pod "ffb64808-c7a9-4880-9a20-db1e53f7e1e2" (UID: "ffb64808-c7a9-4880-9a20-db1e53f7e1e2"). InnerVolumeSpecName "kube-api-access-m7c96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:15:08.603119 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.603092 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.603119 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.603115 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7c96\" (UniqueName: \"kubernetes.io/projected/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-kube-api-access-m7c96\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.603253 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.603126 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.603253 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.603135 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb64808-c7a9-4880-9a20-db1e53f7e1e2-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:08.854851 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.854830 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:15:08.864188 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:08.864170 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:15:09.051624 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.051542 2560 generic.go:356] "Generic (PLEG): container finished" podID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerID="84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12" exitCode=0 Jul 18 03:15:09.051624 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.051583 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" Jul 18 03:15:09.051807 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.051620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" event={"ID":"ffb64808-c7a9-4880-9a20-db1e53f7e1e2","Type":"ContainerDied","Data":"84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12"} Jul 18 03:15:09.051807 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.051673 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s" event={"ID":"ffb64808-c7a9-4880-9a20-db1e53f7e1e2","Type":"ContainerDied","Data":"585f516446771bea9113dba796739ca7d645bc7003d9f60bb551b03469179989"} Jul 18 03:15:09.051807 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.051695 2560 scope.go:117] "RemoveContainer" containerID="84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12" Jul 18 03:15:09.060745 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.060726 2560 scope.go:117] "RemoveContainer" containerID="fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86" Jul 18 03:15:09.069146 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.069123 2560 scope.go:117] "RemoveContainer" containerID="84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12" Jul 18 03:15:09.069416 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:15:09.069397 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12\": container with ID starting with 84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12 not found: ID does not exist" containerID="84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12" Jul 18 03:15:09.069471 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.069423 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12"} err="failed to get container status \"84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12\": rpc error: code = NotFound desc = could not find container \"84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12\": container with ID starting with 84a4a6103cc3e6d729917c63df100410da090ad414ef3646a2b1813fe24c1e12 not found: ID does not exist" Jul 18 03:15:09.069471 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.069441 2560 scope.go:117] "RemoveContainer" containerID="fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86" Jul 18 03:15:09.069702 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:15:09.069685 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86\": container with ID starting with fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86 not found: ID does not exist" containerID="fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86" Jul 18 03:15:09.069745 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.069709 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86"} err="failed to get container status \"fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86\": rpc error: code = NotFound desc = could not find container \"fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86\": container with ID starting with fe161bae24b5e07707d91379bb26df0152b051e3a7490485d89b217b4e452c86 not found: ID does not exist" Jul 18 03:15:09.071603 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.071584 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s"] Jul 18 03:15:09.075510 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:09.075479 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-69f74a-openshift-default-7d7dfc8d9f-wp29s"] Jul 18 03:15:10.757835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:10.757790 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" path="/var/lib/kubelet/pods/ffb64808-c7a9-4880-9a20-db1e53f7e1e2/volumes" Jul 18 03:15:25.170899 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:25.170865 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72"] Jul 18 03:15:25.171390 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:25.171101 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" containerID="cri-o://f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f" gracePeriod=30 Jul 18 03:15:26.079863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.079830 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp"] Jul 18 03:15:26.080221 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.080207 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" Jul 18 03:15:26.080221 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.080222 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" Jul 18 03:15:26.080317 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.080238 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" Jul 18 03:15:26.080317 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.080243 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" Jul 18 03:15:26.080317 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.080301 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" Jul 18 03:15:26.080317 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.080310 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb64808-c7a9-4880-9a20-db1e53f7e1e2" containerName="istio-proxy" Jul 18 03:15:26.082363 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.082347 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.085229 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.085207 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4e7186-openshift-default-dockercfg-25zmz\"" Jul 18 03:15:26.099642 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.099618 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp"] Jul 18 03:15:26.258474 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-certs\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-data\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istiod-ca-cert\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258579 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-credential-socket\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnc9g\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-kube-api-access-tnc9g\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-token\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-envoy\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-socket\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.258863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.258780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-podinfo\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360123 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360055 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-credential-socket\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360123 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnc9g\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-kube-api-access-tnc9g\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-token\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360181 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-envoy\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-socket\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-podinfo\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-certs\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360558 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-data\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360558 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istiod-ca-cert\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360558 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360512 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-credential-socket\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360611 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-socket\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360807 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-certs\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.360872 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.360827 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-data\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.361162 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.361144 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istiod-ca-cert\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.362418 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.362403 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-envoy\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.362726 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.362707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-podinfo\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.368933 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.368910 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-token\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.370998 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.370961 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnc9g\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-kube-api-access-tnc9g\") pod \"e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.395941 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.395918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:26.557198 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.557173 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp"] Jul 18 03:15:26.558141 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:15:26.558092 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1477c8b9_4d60_4572_b52c_82d6a88bf90d.slice/crio-788d7b661a2f83a8d6dba623f384d17acfa0c55d60c0ef416a885e5aff75fc32 WatchSource:0}: Error finding container 788d7b661a2f83a8d6dba623f384d17acfa0c55d60c0ef416a885e5aff75fc32: Status 404 returned error can't find the container with id 788d7b661a2f83a8d6dba623f384d17acfa0c55d60c0ef416a885e5aff75fc32 Jul 18 03:15:26.560240 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.560209 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:26.560309 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.560271 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:26.560367 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:26.560305 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:27.150924 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:27.150843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" event={"ID":"1477c8b9-4d60-4572-b52c-82d6a88bf90d","Type":"ContainerStarted","Data":"4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227"} Jul 18 03:15:27.151121 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:27.150952 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" event={"ID":"1477c8b9-4d60-4572-b52c-82d6a88bf90d","Type":"ContainerStarted","Data":"788d7b661a2f83a8d6dba623f384d17acfa0c55d60c0ef416a885e5aff75fc32"} Jul 18 03:15:27.174984 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:27.174930 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podStartSLOduration=1.174914661 podStartE2EDuration="1.174914661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:15:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:15:27.173351703 +0000 UTC m=+2118.962125476" watchObservedRunningTime="2026-07-18 03:15:27.174914661 +0000 UTC m=+2118.963688431" Jul 18 03:15:27.396810 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:27.396777 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:27.401625 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:27.401565 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:28.156439 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:28.156402 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:28.157236 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:28.157219 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:29.031748 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:29.031714 2560 patch_prober.go:28] interesting pod/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 03:15:29.032166 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:29.031773 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 03:15:30.422550 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.422527 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:15:30.601179 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601152 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-envoy\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601323 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601211 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istiod-ca-cert\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601323 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601241 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-socket\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601323 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601278 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-credential-socket\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601506 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601332 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-podinfo\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601506 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601361 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-certs\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601506 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601399 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxtsp\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-kube-api-access-cxtsp\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601506 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601436 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-data\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601506 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601498 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-token\") pod \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\" (UID: \"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878\") " Jul 18 03:15:30.601754 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601573 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:30.601754 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601646 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:15:30.601864 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601803 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:30.601926 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601914 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.601999 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601932 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.601999 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.601945 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.602368 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.602336 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:30.602477 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.602447 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:30.603672 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.603642 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:15:30.603776 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.603703 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:15:30.603841 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.603821 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:15:30.603901 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.603871 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-kube-api-access-cxtsp" (OuterVolumeSpecName: "kube-api-access-cxtsp") pod "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" (UID: "a2ba8e0d-9c16-48ee-9ae7-afa8915a7878"). InnerVolumeSpecName "kube-api-access-cxtsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:15:30.702621 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.702588 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.702621 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.702616 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.702799 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.702629 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.702799 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.702642 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.702799 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.702654 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxtsp\" (UniqueName: \"kubernetes.io/projected/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-kube-api-access-cxtsp\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:30.702799 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:30.702664 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:15:31.175770 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.175735 2560 generic.go:356] "Generic (PLEG): container finished" podID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerID="f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f" exitCode=0 Jul 18 03:15:31.175928 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.175784 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" Jul 18 03:15:31.175928 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.175787 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" event={"ID":"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878","Type":"ContainerDied","Data":"f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f"} Jul 18 03:15:31.175928 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.175838 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72" event={"ID":"a2ba8e0d-9c16-48ee-9ae7-afa8915a7878","Type":"ContainerDied","Data":"cb9e2b461693c463902aba09e5102e94604733cae84997a8a486568f5edfd62b"} Jul 18 03:15:31.175928 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.175868 2560 scope.go:117] "RemoveContainer" containerID="f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f" Jul 18 03:15:31.185061 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.185042 2560 scope.go:117] "RemoveContainer" containerID="4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0" Jul 18 03:15:31.192947 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.192931 2560 scope.go:117] "RemoveContainer" containerID="f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f" Jul 18 03:15:31.193256 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:15:31.193235 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f\": container with ID starting with f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f not found: ID does not exist" containerID="f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f" Jul 18 03:15:31.193352 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.193272 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f"} err="failed to get container status \"f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f\": rpc error: code = NotFound desc = could not find container \"f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f\": container with ID starting with f06d188202af07ddaf2977b341c44b1ab0efd2c3318eb896508f87d9f01a534f not found: ID does not exist" Jul 18 03:15:31.193352 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.193297 2560 scope.go:117] "RemoveContainer" containerID="4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0" Jul 18 03:15:31.193633 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:15:31.193612 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0\": container with ID starting with 4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0 not found: ID does not exist" containerID="4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0" Jul 18 03:15:31.193701 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.193643 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0"} err="failed to get container status \"4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0\": rpc error: code = NotFound desc = could not find container \"4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0\": container with ID starting with 4f958edabd112761a1c8fc6bcc99ebbc62b2a0cd0191a5b3d2e9c38cfe9b4bf0 not found: ID does not exist" Jul 18 03:15:31.196690 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.196666 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72"] Jul 18 03:15:31.199046 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:31.199026 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-cde1c8-openshift-default-f9c8d565f-22x72"] Jul 18 03:15:32.762578 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:32.762544 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" path="/var/lib/kubelet/pods/a2ba8e0d-9c16-48ee-9ae7-afa8915a7878/volumes" Jul 18 03:15:34.403129 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403097 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275"] Jul 18 03:15:34.403704 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403516 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" Jul 18 03:15:34.403704 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403558 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" Jul 18 03:15:34.403704 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403594 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" Jul 18 03:15:34.403704 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403604 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" Jul 18 03:15:34.403704 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403697 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" Jul 18 03:15:34.403704 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.403706 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ba8e0d-9c16-48ee-9ae7-afa8915a7878" containerName="istio-proxy" Jul 18 03:15:34.406817 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.406798 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.409821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.409803 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-bddfdd-openshift-default-dockercfg-8l8tx\"" Jul 18 03:15:34.419640 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.419618 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275"] Jul 18 03:15:34.438995 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.438949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-certs\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449409 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-token\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449670 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-envoy\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449823 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449802 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-podinfo\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449962 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-credential-socket\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449962 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449878 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3810b35d-ccec-4f70-b16e-d2e677641ed5-istiod-ca-cert\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449962 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449931 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-socket\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.449962 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.449957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qt5f\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-kube-api-access-6qt5f\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.450241 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.450074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-data\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550579 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-podinfo\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550579 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-credential-socket\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3810b35d-ccec-4f70-b16e-d2e677641ed5-istiod-ca-cert\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-socket\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550647 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qt5f\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-kube-api-access-6qt5f\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-data\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-certs\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.550821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-token\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.551182 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.550836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-envoy\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.551182 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.551118 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-credential-socket\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.551380 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.551355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-socket\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.551518 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.551492 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-data\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.551639 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.551618 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-certs\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.551713 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.551656 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3810b35d-ccec-4f70-b16e-d2e677641ed5-istiod-ca-cert\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.553253 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.553230 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-podinfo\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.553383 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.553355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-envoy\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.559346 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.559313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qt5f\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-kube-api-access-6qt5f\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.559555 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.559537 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-token\") pod \"e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.717388 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.717319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:34.862084 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.862057 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275"] Jul 18 03:15:34.862986 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:15:34.862929 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3810b35d_ccec_4f70_b16e_d2e677641ed5.slice/crio-c88787f35931a90351297b1f9c911a8ec1c3b369a26b0a9c6b083c78a5bb78e2 WatchSource:0}: Error finding container c88787f35931a90351297b1f9c911a8ec1c3b369a26b0a9c6b083c78a5bb78e2: Status 404 returned error can't find the container with id c88787f35931a90351297b1f9c911a8ec1c3b369a26b0a9c6b083c78a5bb78e2 Jul 18 03:15:34.865021 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.864986 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:34.865116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.865050 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:34.865116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:34.865077 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:35.200504 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:35.200463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" event={"ID":"3810b35d-ccec-4f70-b16e-d2e677641ed5","Type":"ContainerStarted","Data":"2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f"} Jul 18 03:15:35.200727 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:35.200533 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" event={"ID":"3810b35d-ccec-4f70-b16e-d2e677641ed5","Type":"ContainerStarted","Data":"c88787f35931a90351297b1f9c911a8ec1c3b369a26b0a9c6b083c78a5bb78e2"} Jul 18 03:15:35.272567 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:35.272518 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podStartSLOduration=1.272503133 podStartE2EDuration="1.272503133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:15:35.271751991 +0000 UTC m=+2127.060525760" watchObservedRunningTime="2026-07-18 03:15:35.272503133 +0000 UTC m=+2127.061276904" Jul 18 03:15:35.718445 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:35.718416 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:35.723229 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:35.723202 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:36.206183 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:36.206151 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:36.207183 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:36.207166 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:44.198113 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:44.198078 2560 patch_prober.go:28] interesting pod/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 03:15:44.198611 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:44.198187 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 03:15:45.263136 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:45.263102 2560 generic.go:356] "Generic (PLEG): container finished" podID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerID="4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227" exitCode=0 Jul 18 03:15:45.263546 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:45.263154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" event={"ID":"1477c8b9-4d60-4572-b52c-82d6a88bf90d","Type":"ContainerDied","Data":"4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227"} Jul 18 03:15:45.263546 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:45.263504 2560 scope.go:117] "RemoveContainer" containerID="4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227" Jul 18 03:15:45.264545 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:45.264512 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:45.264632 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:45.264585 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:45.264632 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:45.264621 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:46.271408 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:46.271368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" event={"ID":"1477c8b9-4d60-4572-b52c-82d6a88bf90d","Type":"ContainerStarted","Data":"6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580"} Jul 18 03:15:46.397046 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:46.397010 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:46.397314 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:46.397287 2560 patch_prober.go:28] interesting pod/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 18 03:15:46.397402 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:46.397376 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 18 03:15:47.397286 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:47.397243 2560 patch_prober.go:28] interesting pod/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 18 03:15:47.397696 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:47.397309 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 18 03:15:48.396503 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:48.396466 2560 patch_prober.go:28] interesting pod/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 18 03:15:48.396716 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:48.396529 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 18 03:15:49.397201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:49.397165 2560 patch_prober.go:28] interesting pod/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 18 03:15:49.397616 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:49.397233 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 18 03:15:50.400092 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:50.400061 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:50.400510 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:50.400362 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:50.401079 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:50.401063 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:15:52.248149 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:52.248106 2560 patch_prober.go:28] interesting pod/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 03:15:52.248537 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:52.248167 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 03:15:54.328088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:54.328053 2560 generic.go:356] "Generic (PLEG): container finished" podID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerID="2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f" exitCode=0 Jul 18 03:15:54.328481 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:54.328129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" event={"ID":"3810b35d-ccec-4f70-b16e-d2e677641ed5","Type":"ContainerDied","Data":"2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f"} Jul 18 03:15:54.328524 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:54.328506 2560 scope.go:117] "RemoveContainer" containerID="2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f" Jul 18 03:15:54.329488 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:54.329462 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:54.329545 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:54.329524 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:54.329584 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:54.329553 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:15:55.337691 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:55.337648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" event={"ID":"3810b35d-ccec-4f70-b16e-d2e677641ed5","Type":"ContainerStarted","Data":"ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab"} Jul 18 03:15:55.718353 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:55.718260 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:55.718533 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:55.718428 2560 patch_prober.go:28] interesting pod/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 18 03:15:55.718533 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:55.718484 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 18 03:15:56.717940 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:56.717903 2560 patch_prober.go:28] interesting pod/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 18 03:15:56.718355 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:56.717957 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 18 03:15:57.718525 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:57.718488 2560 patch_prober.go:28] interesting pod/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 18 03:15:57.718919 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:57.718540 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 18 03:15:58.718096 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:58.718058 2560 patch_prober.go:28] interesting pod/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 18 03:15:58.718273 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:58.718113 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 18 03:15:59.721960 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:59.721927 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:59.722473 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:59.722348 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:15:59.723027 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:15:59.722999 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:16:16.566822 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:16.566788 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp"] Jul 18 03:16:16.567315 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:16.567012 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" containerID="cri-o://6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580" gracePeriod=30 Jul 18 03:16:20.402716 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:20.402671 2560 patch_prober.go:28] interesting pod/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 03:16:20.403120 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:20.402749 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 03:16:21.817681 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.817658 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:16:21.896173 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896111 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnc9g\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-kube-api-access-tnc9g\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896173 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896161 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-envoy\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896461 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896192 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-token\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896461 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896309 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istiod-ca-cert\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896461 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896427 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-podinfo\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896622 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896481 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-socket\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896622 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896526 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-certs\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896622 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896564 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-credential-socket\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896622 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896597 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-data\") pod \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\" (UID: \"1477c8b9-4d60-4572-b52c-82d6a88bf90d\") " Jul 18 03:16:21.896822 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896763 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:21.896822 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896777 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:21.896822 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896680 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:16:21.897098 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.896994 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:21.897098 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.897049 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.897098 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.897070 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.897098 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.897082 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.897609 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.897588 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-data" (OuterVolumeSpecName: "istio-data") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:21.898522 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.898499 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:16:21.898616 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.898537 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:21.898729 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.898712 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:16:21.898842 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.898825 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-kube-api-access-tnc9g" (OuterVolumeSpecName: "kube-api-access-tnc9g") pod "1477c8b9-4d60-4572-b52c-82d6a88bf90d" (UID: "1477c8b9-4d60-4572-b52c-82d6a88bf90d"). InnerVolumeSpecName "kube-api-access-tnc9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:16:21.998428 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.998402 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.998428 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.998426 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.998428 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.998435 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnc9g\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-kube-api-access-tnc9g\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.998648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.998444 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.998648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.998453 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:21.998648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:21.998461 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1477c8b9-4d60-4572-b52c-82d6a88bf90d-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:22.491674 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.491632 2560 generic.go:356] "Generic (PLEG): container finished" podID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerID="6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580" exitCode=0 Jul 18 03:16:22.491876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.491673 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" event={"ID":"1477c8b9-4d60-4572-b52c-82d6a88bf90d","Type":"ContainerDied","Data":"6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580"} Jul 18 03:16:22.491876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.491744 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" event={"ID":"1477c8b9-4d60-4572-b52c-82d6a88bf90d","Type":"ContainerDied","Data":"788d7b661a2f83a8d6dba623f384d17acfa0c55d60c0ef416a885e5aff75fc32"} Jul 18 03:16:22.491876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.491688 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp" Jul 18 03:16:22.491876 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.491776 2560 scope.go:117] "RemoveContainer" containerID="6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580" Jul 18 03:16:22.502344 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.502321 2560 scope.go:117] "RemoveContainer" containerID="4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227" Jul 18 03:16:22.510394 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.510379 2560 scope.go:117] "RemoveContainer" containerID="6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580" Jul 18 03:16:22.510651 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:16:22.510630 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580\": container with ID starting with 6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580 not found: ID does not exist" containerID="6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580" Jul 18 03:16:22.510698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.510661 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580"} err="failed to get container status \"6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580\": rpc error: code = NotFound desc = could not find container \"6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580\": container with ID starting with 6eff7f860f8e190e62d2a92938a9bce61219461db9915591f98a9c5aa8152580 not found: ID does not exist" Jul 18 03:16:22.510698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.510678 2560 scope.go:117] "RemoveContainer" containerID="4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227" Jul 18 03:16:22.510951 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:16:22.510933 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227\": container with ID starting with 4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227 not found: ID does not exist" containerID="4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227" Jul 18 03:16:22.511009 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.510961 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227"} err="failed to get container status \"4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227\": rpc error: code = NotFound desc = could not find container \"4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227\": container with ID starting with 4bf669dbb9b3c09e3eb529031af42aa25c2efcf22fd9861b6e66dec29fa08227 not found: ID does not exist" Jul 18 03:16:22.515526 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.515507 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp"] Jul 18 03:16:22.521797 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.521775 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4e7186-openshift-default-688cfb9877-ksfpp"] Jul 18 03:16:22.763551 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:22.763522 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" path="/var/lib/kubelet/pods/1477c8b9-4d60-4572-b52c-82d6a88bf90d/volumes" Jul 18 03:16:43.576365 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:43.576321 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275"] Jul 18 03:16:43.576884 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:43.576575 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" containerID="cri-o://ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab" gracePeriod=30 Jul 18 03:16:44.497550 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.497517 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp"] Jul 18 03:16:44.497941 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.497929 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" Jul 18 03:16:44.498009 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.497942 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" Jul 18 03:16:44.498009 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.497953 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" Jul 18 03:16:44.498009 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.497958 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" Jul 18 03:16:44.498108 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.498048 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" Jul 18 03:16:44.498108 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.498056 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1477c8b9-4d60-4572-b52c-82d6a88bf90d" containerName="istio-proxy" Jul 18 03:16:44.500208 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.500192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.503239 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.503220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e88910-openshift-default-dockercfg-klrzd\"" Jul 18 03:16:44.515943 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.515913 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp"] Jul 18 03:16:44.592810 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.592764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-token\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.592860 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-envoy\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.592950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-certs\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.593024 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-data\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.593072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-socket\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.593125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-credential-socket\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.593172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istiod-ca-cert\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.593199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-podinfo\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.593305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.593283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44jbv\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-kube-api-access-44jbv\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694324 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istiod-ca-cert\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694324 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-podinfo\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694561 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44jbv\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-kube-api-access-44jbv\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694561 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-token\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694561 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-envoy\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694561 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-certs\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694561 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694529 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-data\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694820 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694573 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-socket\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694820 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-credential-socket\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.694943 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.694918 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-certs\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.695056 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.695030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-data\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.695164 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.695078 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-socket\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.695164 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.695142 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istiod-ca-cert\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.695268 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.695252 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-credential-socket\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.697203 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.697179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-envoy\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.697375 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.697360 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-podinfo\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.704328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.704308 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-token\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.704576 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.704556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44jbv\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-kube-api-access-44jbv\") pod \"e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.724278 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.724255 2560 patch_prober.go:28] interesting pod/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 03:16:44.724360 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.724309 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 03:16:44.809978 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.809951 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:44.943638 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.943615 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp"] Jul 18 03:16:44.944901 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:16:44.944871 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756db85e_fbb5_4b67_888b_0d4ddbd2fc38.slice/crio-dc518c38c3fe1a84d4cf81307b010e6f738889f84fc121b2a7d6f39729f16b3b WatchSource:0}: Error finding container dc518c38c3fe1a84d4cf81307b010e6f738889f84fc121b2a7d6f39729f16b3b: Status 404 returned error can't find the container with id dc518c38c3fe1a84d4cf81307b010e6f738889f84fc121b2a7d6f39729f16b3b Jul 18 03:16:44.947057 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.947025 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:16:44.947157 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.947089 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:16:44.947157 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:44.947120 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:16:45.619230 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:45.619196 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" event={"ID":"756db85e-fbb5-4b67-888b-0d4ddbd2fc38","Type":"ContainerStarted","Data":"9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee"} Jul 18 03:16:45.619664 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:45.619254 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" event={"ID":"756db85e-fbb5-4b67-888b-0d4ddbd2fc38","Type":"ContainerStarted","Data":"dc518c38c3fe1a84d4cf81307b010e6f738889f84fc121b2a7d6f39729f16b3b"} Jul 18 03:16:45.650826 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:45.650782 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podStartSLOduration=1.650769467 podStartE2EDuration="1.650769467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:16:45.647463205 +0000 UTC m=+2197.436237004" watchObservedRunningTime="2026-07-18 03:16:45.650769467 +0000 UTC m=+2197.439543237" Jul 18 03:16:45.810955 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:45.810926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:46.814699 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:46.814666 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:47.629602 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:47.629574 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:47.630596 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:47.630569 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:16:48.830785 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.830763 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:16:48.935525 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935457 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-socket\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935525 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935487 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-token\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935543 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-envoy\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935572 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-podinfo\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935588 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qt5f\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-kube-api-access-6qt5f\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935656 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-credential-socket\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935698 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935683 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3810b35d-ccec-4f70-b16e-d2e677641ed5-istiod-ca-cert\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935960 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935725 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-data\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.935960 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935744 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:48.935960 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.935765 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-certs\") pod \"3810b35d-ccec-4f70-b16e-d2e677641ed5\" (UID: \"3810b35d-ccec-4f70-b16e-d2e677641ed5\") " Jul 18 03:16:48.936238 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.936208 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:48.936330 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.936237 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:48.936439 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.936411 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:48.936621 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.936598 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3810b35d-ccec-4f70-b16e-d2e677641ed5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:16:48.936904 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.936877 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-data" (OuterVolumeSpecName: "istio-data") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:48.937821 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.937789 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-token" (OuterVolumeSpecName: "istio-token") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:16:48.937942 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.937926 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-kube-api-access-6qt5f" (OuterVolumeSpecName: "kube-api-access-6qt5f") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "kube-api-access-6qt5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:16:48.938192 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.938168 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:16:48.938312 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:48.938292 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3810b35d-ccec-4f70-b16e-d2e677641ed5" (UID: "3810b35d-ccec-4f70-b16e-d2e677641ed5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:16:49.037319 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037296 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037319 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037318 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3810b35d-ccec-4f70-b16e-d2e677641ed5-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037467 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037327 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037467 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037335 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037467 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037342 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037467 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037349 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037467 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037356 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3810b35d-ccec-4f70-b16e-d2e677641ed5-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.037467 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.037363 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qt5f\" (UniqueName: \"kubernetes.io/projected/3810b35d-ccec-4f70-b16e-d2e677641ed5-kube-api-access-6qt5f\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:16:49.641984 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.641944 2560 generic.go:356] "Generic (PLEG): container finished" podID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerID="ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab" exitCode=0 Jul 18 03:16:49.642173 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.642038 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" Jul 18 03:16:49.642173 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.642037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" event={"ID":"3810b35d-ccec-4f70-b16e-d2e677641ed5","Type":"ContainerDied","Data":"ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab"} Jul 18 03:16:49.642296 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.642190 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275" event={"ID":"3810b35d-ccec-4f70-b16e-d2e677641ed5","Type":"ContainerDied","Data":"c88787f35931a90351297b1f9c911a8ec1c3b369a26b0a9c6b083c78a5bb78e2"} Jul 18 03:16:49.642296 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.642215 2560 scope.go:117] "RemoveContainer" containerID="ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab" Jul 18 03:16:49.651797 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.651776 2560 scope.go:117] "RemoveContainer" containerID="2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f" Jul 18 03:16:49.659787 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.659769 2560 scope.go:117] "RemoveContainer" containerID="ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab" Jul 18 03:16:49.660049 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:16:49.660033 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab\": container with ID starting with ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab not found: ID does not exist" containerID="ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab" Jul 18 03:16:49.660103 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.660056 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab"} err="failed to get container status \"ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab\": rpc error: code = NotFound desc = could not find container \"ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab\": container with ID starting with ff5766545e49ad86a842ab987f4ef04be60b5f926c229b4313ebff1d09bd75ab not found: ID does not exist" Jul 18 03:16:49.660103 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.660071 2560 scope.go:117] "RemoveContainer" containerID="2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f" Jul 18 03:16:49.660307 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:16:49.660290 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f\": container with ID starting with 2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f not found: ID does not exist" containerID="2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f" Jul 18 03:16:49.660356 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.660311 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f"} err="failed to get container status \"2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f\": rpc error: code = NotFound desc = could not find container \"2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f\": container with ID starting with 2c5790974c39013a07f6c490af315f2f9d8700133a1aeb790ba6f20d18f1868f not found: ID does not exist" Jul 18 03:16:49.671964 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.671942 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275"] Jul 18 03:16:49.677282 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:49.677254 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-bddfdd-openshift-default-849bb5ff7-km275"] Jul 18 03:16:50.758612 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:50.758581 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" path="/var/lib/kubelet/pods/3810b35d-ccec-4f70-b16e-d2e677641ed5/volumes" Jul 18 03:16:52.739256 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739225 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw"] Jul 18 03:16:52.739808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739588 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" Jul 18 03:16:52.739808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739599 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" Jul 18 03:16:52.739808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739611 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" Jul 18 03:16:52.739808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739616 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" Jul 18 03:16:52.739808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739685 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" Jul 18 03:16:52.739808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.739692 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3810b35d-ccec-4f70-b16e-d2e677641ed5" containerName="istio-proxy" Jul 18 03:16:52.742841 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.742817 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.745293 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.745268 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b50cfd-openshift-default-dockercfg-4skxf\"" Jul 18 03:16:52.758526 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.758503 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw"] Jul 18 03:16:52.769586 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-data\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769722 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769599 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-certs\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769722 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a487b944-2b40-4a26-a711-9a2283b27ccc-istiod-ca-cert\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769722 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769700 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nwvf\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-kube-api-access-7nwvf\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-credential-socket\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769771 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-podinfo\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769793 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-socket\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-token\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.769907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.769885 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-envoy\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871212 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-socket\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871422 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-token\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871422 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-envoy\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871551 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871411 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-data\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871551 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871456 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-certs\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871551 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a487b944-2b40-4a26-a711-9a2283b27ccc-istiod-ca-cert\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871710 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nwvf\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-kube-api-access-7nwvf\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871710 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871625 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-socket\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871710 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-credential-socket\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871710 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-podinfo\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.871917 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.871738 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-data\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.872185 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.872161 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-certs\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.872301 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.872199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-credential-socket\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.872365 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.872335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a487b944-2b40-4a26-a711-9a2283b27ccc-istiod-ca-cert\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.873753 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.873724 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-envoy\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.874389 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.874368 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-podinfo\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.880118 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.880097 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-token\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:52.880203 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:52.880172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nwvf\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-kube-api-access-7nwvf\") pod \"e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:53.053049 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.053022 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:53.192429 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.192404 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw"] Jul 18 03:16:53.193307 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:16:53.193283 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda487b944_2b40_4a26_a711_9a2283b27ccc.slice/crio-4449e17287d78a660961847c3cc363415eff919ecaff4852c81f05fa0ee4bfc1 WatchSource:0}: Error finding container 4449e17287d78a660961847c3cc363415eff919ecaff4852c81f05fa0ee4bfc1: Status 404 returned error can't find the container with id 4449e17287d78a660961847c3cc363415eff919ecaff4852c81f05fa0ee4bfc1 Jul 18 03:16:53.195813 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.195781 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:16:53.195907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.195847 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:16:53.195907 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.195877 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:16:53.667775 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.667743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" event={"ID":"a487b944-2b40-4a26-a711-9a2283b27ccc","Type":"ContainerStarted","Data":"55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd"} Jul 18 03:16:53.667957 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.667797 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" event={"ID":"a487b944-2b40-4a26-a711-9a2283b27ccc","Type":"ContainerStarted","Data":"4449e17287d78a660961847c3cc363415eff919ecaff4852c81f05fa0ee4bfc1"} Jul 18 03:16:53.691849 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:53.691804 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podStartSLOduration=1.691789886 podStartE2EDuration="1.691789886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 03:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:16:53.688985526 +0000 UTC m=+2205.477759287" watchObservedRunningTime="2026-07-18 03:16:53.691789886 +0000 UTC m=+2205.480563693" Jul 18 03:16:54.053823 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:54.053794 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:54.058682 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:54.058656 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:54.673307 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:54.673272 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:16:54.674101 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:16:54.674077 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:17:03.446797 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.446760 2560 patch_prober.go:28] interesting pod/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": read tcp 10.134.0.2:34000->10.134.0.62:15021: read: connection reset by peer" start-of-body= Jul 18 03:17:03.447240 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.446842 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": read tcp 10.134.0.2:34000->10.134.0.62:15021: read: connection reset by peer" Jul 18 03:17:03.723718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.723613 2560 generic.go:356] "Generic (PLEG): container finished" podID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerID="9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee" exitCode=0 Jul 18 03:17:03.723718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.723689 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" event={"ID":"756db85e-fbb5-4b67-888b-0d4ddbd2fc38","Type":"ContainerDied","Data":"9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee"} Jul 18 03:17:03.724116 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.724098 2560 scope.go:117] "RemoveContainer" containerID="9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee" Jul 18 03:17:03.725232 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.725196 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:17:03.725313 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.725289 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:17:03.725353 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:03.725330 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:17:04.748023 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:04.747899 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" event={"ID":"756db85e-fbb5-4b67-888b-0d4ddbd2fc38","Type":"ContainerStarted","Data":"e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086"} Jul 18 03:17:04.811036 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:04.811004 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:17:04.811255 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:04.811235 2560 patch_prober.go:28] interesting pod/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 18 03:17:04.811320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:04.811278 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 18 03:17:05.810629 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:05.810592 2560 patch_prober.go:28] interesting pod/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 18 03:17:05.811058 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:05.810643 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 18 03:17:06.810624 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:06.810590 2560 patch_prober.go:28] interesting pod/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 18 03:17:06.810791 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:06.810655 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 18 03:17:07.810607 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:07.810568 2560 patch_prober.go:28] interesting pod/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 18 03:17:07.810797 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:07.810640 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 18 03:17:08.813808 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:08.813775 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:17:08.814263 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:08.814010 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:17:08.814678 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:08.814659 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:17:10.610905 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.610558 2560 patch_prober.go:28] interesting pod/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:58970->10.134.0.63:15021: read: connection reset by peer" start-of-body= Jul 18 03:17:10.610905 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.610629 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:58970->10.134.0.63:15021: read: connection reset by peer" Jul 18 03:17:10.794720 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.794654 2560 generic.go:356] "Generic (PLEG): container finished" podID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerID="55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd" exitCode=0 Jul 18 03:17:10.794838 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.794732 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" event={"ID":"a487b944-2b40-4a26-a711-9a2283b27ccc","Type":"ContainerDied","Data":"55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd"} Jul 18 03:17:10.795165 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.795150 2560 scope.go:117] "RemoveContainer" containerID="55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd" Jul 18 03:17:10.796096 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.796070 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:17:10.796165 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.796141 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:17:10.796209 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:10.796174 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 03:17:11.802548 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:11.802517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" event={"ID":"a487b944-2b40-4a26-a711-9a2283b27ccc","Type":"ContainerStarted","Data":"349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f"} Jul 18 03:17:12.053347 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:12.053241 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:17:12.053927 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:12.053618 2560 patch_prober.go:28] interesting pod/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 18 03:17:12.053927 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:12.053671 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 18 03:17:13.053504 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:13.053460 2560 patch_prober.go:28] interesting pod/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 18 03:17:13.054041 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:13.053564 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 18 03:17:14.054235 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:14.054193 2560 patch_prober.go:28] interesting pod/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 18 03:17:14.054753 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:14.054258 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 18 03:17:15.054207 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:15.054170 2560 patch_prober.go:28] interesting pod/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 18 03:17:15.054494 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:15.054464 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 18 03:17:16.058835 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:16.058805 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:17:16.059294 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:16.059278 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:17:16.060007 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:17:16.059959 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:20:08.891201 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:20:08.891175 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:20:08.901239 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:20:08.901218 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4grnx_8c0e45f7-67a2-4ef8-8fa6-37c879bfac64/ovn-acl-logging/0.log" Jul 18 03:23:38.435264 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:38.435183 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp"] Jul 18 03:23:38.435752 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:38.435494 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" containerID="cri-o://e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086" gracePeriod=30 Jul 18 03:23:38.816529 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:38.816495 2560 patch_prober.go:28] interesting pod/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 03:23:38.816716 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:38.816588 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 03:23:43.680745 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.680713 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:23:43.773589 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773561 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istiod-ca-cert\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.773771 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773601 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-certs\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.773771 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773683 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-socket\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.773771 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773706 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-token\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.773771 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773737 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-envoy\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.773771 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773755 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44jbv\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-kube-api-access-44jbv\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.774088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773808 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-podinfo\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.774088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773844 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-credential-socket\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.774088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.773963 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:23:43.774088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774003 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:23:43.774088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774053 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-data\") pod \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\" (UID: \"756db85e-fbb5-4b67-888b-0d4ddbd2fc38\") " Jul 18 03:23:43.774088 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774057 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:23:43.774393 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774156 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:23:43.774550 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774496 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.774550 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774522 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.774550 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774538 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.774550 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774552 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.774893 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.774871 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-data" (OuterVolumeSpecName: "istio-data") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:23:43.776136 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.776113 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:23:43.776240 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.776139 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-kube-api-access-44jbv" (OuterVolumeSpecName: "kube-api-access-44jbv") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "kube-api-access-44jbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:23:43.776348 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.776330 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-token" (OuterVolumeSpecName: "istio-token") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:23:43.776410 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.776342 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "756db85e-fbb5-4b67-888b-0d4ddbd2fc38" (UID: "756db85e-fbb5-4b67-888b-0d4ddbd2fc38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:23:43.875705 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.875663 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.875705 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.875703 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.875705 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.875713 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44jbv\" (UniqueName: \"kubernetes.io/projected/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-kube-api-access-44jbv\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.875944 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.875723 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.875944 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.875731 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/756db85e-fbb5-4b67-888b-0d4ddbd2fc38-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:23:43.947016 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.946961 2560 generic.go:356] "Generic (PLEG): container finished" podID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerID="e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086" exitCode=0 Jul 18 03:23:43.947174 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.947028 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" Jul 18 03:23:43.947174 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.947071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" event={"ID":"756db85e-fbb5-4b67-888b-0d4ddbd2fc38","Type":"ContainerDied","Data":"e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086"} Jul 18 03:23:43.947174 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.947125 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp" event={"ID":"756db85e-fbb5-4b67-888b-0d4ddbd2fc38","Type":"ContainerDied","Data":"dc518c38c3fe1a84d4cf81307b010e6f738889f84fc121b2a7d6f39729f16b3b"} Jul 18 03:23:43.947174 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.947156 2560 scope.go:117] "RemoveContainer" containerID="e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086" Jul 18 03:23:43.956701 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.956682 2560 scope.go:117] "RemoveContainer" containerID="9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee" Jul 18 03:23:43.964667 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.964646 2560 scope.go:117] "RemoveContainer" containerID="e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086" Jul 18 03:23:43.964910 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:23:43.964890 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086\": container with ID starting with e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086 not found: ID does not exist" containerID="e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086" Jul 18 03:23:43.964986 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.964917 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086"} err="failed to get container status \"e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086\": rpc error: code = NotFound desc = could not find container \"e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086\": container with ID starting with e4e72f2094cb83da8042d00ac6bf24508560e2b3eecd19ce6b53d2081226d086 not found: ID does not exist" Jul 18 03:23:43.964986 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.964935 2560 scope.go:117] "RemoveContainer" containerID="9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee" Jul 18 03:23:43.965200 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:23:43.965183 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee\": container with ID starting with 9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee not found: ID does not exist" containerID="9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee" Jul 18 03:23:43.965248 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.965207 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee"} err="failed to get container status \"9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee\": rpc error: code = NotFound desc = could not find container \"9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee\": container with ID starting with 9aa63c83d5f52b8aa704db7e6d56e945eb39e99d5399437ed01bd8a976a4feee not found: ID does not exist" Jul 18 03:23:43.969951 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.969931 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp"] Jul 18 03:23:43.973663 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:43.973642 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e88910-openshift-default-6bd8695f47-lgznp"] Jul 18 03:23:44.758344 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:23:44.758312 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" path="/var/lib/kubelet/pods/756db85e-fbb5-4b67-888b-0d4ddbd2fc38/volumes" Jul 18 03:24:00.282513 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:00.282481 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw"] Jul 18 03:24:00.283073 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:00.282724 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" containerID="cri-o://349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f" gracePeriod=30 Jul 18 03:24:01.061390 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:01.061356 2560 patch_prober.go:28] interesting pod/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 03:24:01.061572 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:01.061418 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 03:24:05.530900 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.530874 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:24:05.573471 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573406 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-socket\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573607 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573478 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-data\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573607 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573520 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a487b944-2b40-4a26-a711-9a2283b27ccc-istiod-ca-cert\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573607 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573572 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-credential-socket\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573609 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:24:05.573784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-certs\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573649 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-envoy\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573686 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nwvf\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-kube-api-access-7nwvf\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.573784 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573748 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-podinfo\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.574065 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573791 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-token\") pod \"a487b944-2b40-4a26-a711-9a2283b27ccc\" (UID: \"a487b944-2b40-4a26-a711-9a2283b27ccc\") " Jul 18 03:24:05.574065 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573863 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:24:05.574065 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573881 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:24:05.574065 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.573956 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a487b944-2b40-4a26-a711-9a2283b27ccc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 03:24:05.574290 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.574178 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a487b944-2b40-4a26-a711-9a2283b27ccc-istiod-ca-cert\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.574290 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.574199 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-credential-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.574290 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.574214 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-certs\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.574290 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.574232 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-workload-socket\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.574552 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.574531 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-data" (OuterVolumeSpecName: "istio-data") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:24:05.576013 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.575989 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 03:24:05.576270 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.576248 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-kube-api-access-7nwvf" (OuterVolumeSpecName: "kube-api-access-7nwvf") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "kube-api-access-7nwvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:24:05.576337 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.576269 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 03:24:05.576381 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.576337 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-token" (OuterVolumeSpecName: "istio-token") pod "a487b944-2b40-4a26-a711-9a2283b27ccc" (UID: "a487b944-2b40-4a26-a711-9a2283b27ccc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 03:24:05.675577 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.675552 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-envoy\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.675577 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.675574 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nwvf\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-kube-api-access-7nwvf\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.675728 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.675583 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-podinfo\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.675728 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.675592 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-token\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:05.675728 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:05.675600 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a487b944-2b40-4a26-a711-9a2283b27ccc-istio-data\") on node \"ip-10-0-138-139.ec2.internal\" DevicePath \"\"" Jul 18 03:24:06.069854 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.069825 2560 generic.go:356] "Generic (PLEG): container finished" podID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerID="349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f" exitCode=0 Jul 18 03:24:06.070047 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.069876 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" Jul 18 03:24:06.070047 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.069902 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" event={"ID":"a487b944-2b40-4a26-a711-9a2283b27ccc","Type":"ContainerDied","Data":"349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f"} Jul 18 03:24:06.070047 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.069982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw" event={"ID":"a487b944-2b40-4a26-a711-9a2283b27ccc","Type":"ContainerDied","Data":"4449e17287d78a660961847c3cc363415eff919ecaff4852c81f05fa0ee4bfc1"} Jul 18 03:24:06.070047 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.070000 2560 scope.go:117] "RemoveContainer" containerID="349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f" Jul 18 03:24:06.079466 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.079450 2560 scope.go:117] "RemoveContainer" containerID="55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd" Jul 18 03:24:06.087320 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.087305 2560 scope.go:117] "RemoveContainer" containerID="349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f" Jul 18 03:24:06.087596 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:24:06.087578 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f\": container with ID starting with 349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f not found: ID does not exist" containerID="349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f" Jul 18 03:24:06.087660 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.087603 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f"} err="failed to get container status \"349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f\": rpc error: code = NotFound desc = could not find container \"349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f\": container with ID starting with 349d21618ff7fce7616a133576d05a4977afe43e40351dd19ddfa173bd9be48f not found: ID does not exist" Jul 18 03:24:06.087660 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.087621 2560 scope.go:117] "RemoveContainer" containerID="55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd" Jul 18 03:24:06.087832 ip-10-0-138-139 kubenswrapper[2560]: E0718 03:24:06.087814 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd\": container with ID starting with 55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd not found: ID does not exist" containerID="55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd" Jul 18 03:24:06.087873 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.087838 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd"} err="failed to get container status \"55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd\": rpc error: code = NotFound desc = could not find container \"55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd\": container with ID starting with 55013cd114852334dc0eee3d709876d0a482cd7a4b2adf80d88adc045f34e3cd not found: ID does not exist" Jul 18 03:24:06.093401 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.093378 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw"] Jul 18 03:24:06.097716 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.097694 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b50cfd-openshift-default-f987f5cd7-9d8bw"] Jul 18 03:24:06.349626 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.349548 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hskm5_748f4be0-e25a-4abb-96b1-44599411a0a3/manager/0.log" Jul 18 03:24:06.614891 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.614818 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d6d58f489-m6grn_1284e775-8b46-4e7d-aa84-3b59ebaa1de8/manager/0.log" Jul 18 03:24:06.724261 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.724231 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5nzlm_6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea/server/0.log" Jul 18 03:24:06.757251 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:06.757207 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" path="/var/lib/kubelet/pods/a487b944-2b40-4a26-a711-9a2283b27ccc/volumes" Jul 18 03:24:07.202476 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:07.202453 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f9896c96c-f4klz_4bfd634f-8e3b-4236-9659-bdd5a2ea9655/copy-manifests/0.log" Jul 18 03:24:07.221716 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:07.221688 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f9896c96c-f4klz_4bfd634f-8e3b-4236-9659-bdd5a2ea9655/manager/0.log" Jul 18 03:24:07.694748 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:07.694714 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739060-sj7mx_c8acdb97-cf72-4bfe-aea3-57482a1068a8/cleanup/0.log" Jul 18 03:24:08.683175 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.683144 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz_b18832bc-eed9-41f8-974f-5ae7d02aa3b2/util/0.log" Jul 18 03:24:08.689363 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.689337 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz_b18832bc-eed9-41f8-974f-5ae7d02aa3b2/pull/0.log" Jul 18 03:24:08.694989 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.694954 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz_b18832bc-eed9-41f8-974f-5ae7d02aa3b2/extract/0.log" Jul 18 03:24:08.812791 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.812769 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl_c9ce62a1-c705-4621-a97e-85775e90df52/util/0.log" Jul 18 03:24:08.819045 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.819025 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl_c9ce62a1-c705-4621-a97e-85775e90df52/pull/0.log" Jul 18 03:24:08.824932 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.824906 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl_c9ce62a1-c705-4621-a97e-85775e90df52/extract/0.log" Jul 18 03:24:08.938795 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.938736 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g_dd1266ca-1d8a-45f3-92e8-25050d26301e/util/0.log" Jul 18 03:24:08.944892 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.944871 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g_dd1266ca-1d8a-45f3-92e8-25050d26301e/pull/0.log" Jul 18 03:24:08.950694 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:08.950678 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g_dd1266ca-1d8a-45f3-92e8-25050d26301e/extract/0.log" Jul 18 03:24:09.063291 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:09.063269 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq_ab4213b8-47d0-473a-a9b8-19a4f91fcc3b/util/0.log" Jul 18 03:24:09.069396 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:09.069378 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq_ab4213b8-47d0-473a-a9b8-19a4f91fcc3b/pull/0.log" Jul 18 03:24:09.075087 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:09.075063 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq_ab4213b8-47d0-473a-a9b8-19a4f91fcc3b/extract/0.log" Jul 18 03:24:09.318450 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:09.318426 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lxd8c_121c571f-0211-4e7c-8cca-5a7a4064ab06/manager/0.log" Jul 18 03:24:09.555329 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:09.555304 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dng5q_7b69b593-57b4-47d5-bc36-426f20c8570f/kuadrant-console-plugin/0.log" Jul 18 03:24:10.382289 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:10.382253 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdswkkg_f55562f7-6c54-4e59-b9e7-0b27e71de895/istio-proxy/0.log" Jul 18 03:24:10.854240 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:10.854210 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/1.log" Jul 18 03:24:10.974384 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:10.974356 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xqsfl_8516c60e-702f-4312-8850-40dac347bfe5/payload-pre-processing/0.log" Jul 18 03:24:11.104445 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:11.104372 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vzgtk_f30942d9-047a-4f67-91e3-f229658a28ea/payload-processing/0.log" Jul 18 03:24:11.808044 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:11.808009 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ngfxk_8a4852d8-442b-4f69-bb30-967abc080b18/main/0.log" Jul 18 03:24:11.919663 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:11.919635 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp5h9h_e4cf117c-214b-43e0-84f3-b39df8358b7c/main/0.log" Jul 18 03:24:20.714202 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:20.714156 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nswl4_ba51b136-217a-436d-80ba-8b5332c264fd/global-pull-secret-syncer/0.log" Jul 18 03:24:20.892039 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:20.892010 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mqgz7_4279194f-e379-4c62-8447-90668c667b97/konnectivity-agent/0.log" Jul 18 03:24:20.941039 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:20.941009 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-139.ec2.internal_1439b4208df4c574b43f6dbf41e494ee/haproxy/0.log" Jul 18 03:24:25.292739 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.292709 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz_b18832bc-eed9-41f8-974f-5ae7d02aa3b2/extract/0.log" Jul 18 03:24:25.318190 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.318164 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz_b18832bc-eed9-41f8-974f-5ae7d02aa3b2/util/0.log" Jul 18 03:24:25.336824 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.336801 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75994mzz_b18832bc-eed9-41f8-974f-5ae7d02aa3b2/pull/0.log" Jul 18 03:24:25.365143 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.365123 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl_c9ce62a1-c705-4621-a97e-85775e90df52/extract/0.log" Jul 18 03:24:25.383668 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.383647 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl_c9ce62a1-c705-4621-a97e-85775e90df52/util/0.log" Jul 18 03:24:25.405749 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.405731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l7btl_c9ce62a1-c705-4621-a97e-85775e90df52/pull/0.log" Jul 18 03:24:25.431288 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.431269 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g_dd1266ca-1d8a-45f3-92e8-25050d26301e/extract/0.log" Jul 18 03:24:25.454874 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.454858 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g_dd1266ca-1d8a-45f3-92e8-25050d26301e/util/0.log" Jul 18 03:24:25.475575 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.475554 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73w8s5g_dd1266ca-1d8a-45f3-92e8-25050d26301e/pull/0.log" Jul 18 03:24:25.498511 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.498494 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq_ab4213b8-47d0-473a-a9b8-19a4f91fcc3b/extract/0.log" Jul 18 03:24:25.516569 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.516553 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq_ab4213b8-47d0-473a-a9b8-19a4f91fcc3b/util/0.log" Jul 18 03:24:25.535309 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:25.535285 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdbfq_ab4213b8-47d0-473a-a9b8-19a4f91fcc3b/pull/0.log" Jul 18 03:24:26.053862 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:26.053832 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lxd8c_121c571f-0211-4e7c-8cca-5a7a4064ab06/manager/0.log" Jul 18 03:24:26.110442 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:26.110416 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dng5q_7b69b593-57b4-47d5-bc36-426f20c8570f/kuadrant-console-plugin/0.log" Jul 18 03:24:28.097750 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.097721 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/alertmanager/0.log" Jul 18 03:24:28.115983 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.115948 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/config-reloader/0.log" Jul 18 03:24:28.134676 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.134649 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/kube-rbac-proxy-web/0.log" Jul 18 03:24:28.164303 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.164285 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/kube-rbac-proxy/0.log" Jul 18 03:24:28.185718 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.185698 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/kube-rbac-proxy-metric/0.log" Jul 18 03:24:28.205064 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.205045 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/prom-label-proxy/0.log" Jul 18 03:24:28.226689 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.226671 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2135806a-4f69-4cb9-9f03-ffc022effd95/init-config-reloader/0.log" Jul 18 03:24:28.366850 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.366776 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-89dc48964-lm9rd_fe81a410-7b19-4517-bb67-548da9268822/metrics-server/0.log" Jul 18 03:24:28.388844 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.388814 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-k5p2r_25336ac5-f4c9-49f8-92a4-d5c93a5686da/monitoring-plugin/0.log" Jul 18 03:24:28.501934 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.501911 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kfqqj_56c07650-f10f-4901-ab0e-bf61796f6fe4/node-exporter/0.log" Jul 18 03:24:28.518314 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.518297 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kfqqj_56c07650-f10f-4901-ab0e-bf61796f6fe4/kube-rbac-proxy/0.log" Jul 18 03:24:28.537212 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.537189 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kfqqj_56c07650-f10f-4901-ab0e-bf61796f6fe4/init-textfile/0.log" Jul 18 03:24:28.903152 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.903127 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-nfvp9_cf60079b-1b6e-42a5-bc0f-af53a11090dd/prometheus-operator/0.log" Jul 18 03:24:28.922531 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.922508 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-nfvp9_cf60079b-1b6e-42a5-bc0f-af53a11090dd/kube-rbac-proxy/0.log" Jul 18 03:24:28.976543 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.976519 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7595c577bc-pxcq6_fc3fb636-0a27-43fc-b8a1-757ac248cbcb/telemeter-client/0.log" Jul 18 03:24:28.996155 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:28.996126 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7595c577bc-pxcq6_fc3fb636-0a27-43fc-b8a1-757ac248cbcb/reload/0.log" Jul 18 03:24:29.014328 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:29.014308 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7595c577bc-pxcq6_fc3fb636-0a27-43fc-b8a1-757ac248cbcb/kube-rbac-proxy/0.log" Jul 18 03:24:31.183289 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183253 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5"] Jul 18 03:24:31.183803 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183787 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" Jul 18 03:24:31.183863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183807 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" Jul 18 03:24:31.183863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183827 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" Jul 18 03:24:31.183863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183835 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" Jul 18 03:24:31.183863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183846 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" Jul 18 03:24:31.183863 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183852 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" Jul 18 03:24:31.184059 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183866 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" Jul 18 03:24:31.184059 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183871 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" Jul 18 03:24:31.184059 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183931 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" Jul 18 03:24:31.184059 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183940 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a487b944-2b40-4a26-a711-9a2283b27ccc" containerName="istio-proxy" Jul 18 03:24:31.184059 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.183949 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="756db85e-fbb5-4b67-888b-0d4ddbd2fc38" containerName="istio-proxy" Jul 18 03:24:31.186004 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.185990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.188476 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.188451 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tn4zf\"/\"default-dockercfg-glwsz\"" Jul 18 03:24:31.188585 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.188460 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tn4zf\"/\"kube-root-ca.crt\"" Jul 18 03:24:31.189270 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.189255 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tn4zf\"/\"openshift-service-ca.crt\"" Jul 18 03:24:31.197165 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.197145 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5"] Jul 18 03:24:31.293612 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.293581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsv8v\" (UniqueName: \"kubernetes.io/projected/6750ba0a-2d11-4cff-a836-ff02a315a8f9-kube-api-access-gsv8v\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.293782 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.293706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-proc\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.293782 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.293771 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-sys\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.293891 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.293805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-lib-modules\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.293891 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.293860 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-podres\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394403 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-proc\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-sys\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-lib-modules\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-podres\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394502 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-proc\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsv8v\" (UniqueName: \"kubernetes.io/projected/6750ba0a-2d11-4cff-a836-ff02a315a8f9-kube-api-access-gsv8v\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-sys\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394619 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-podres\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.394648 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.394632 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6750ba0a-2d11-4cff-a836-ff02a315a8f9-lib-modules\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.402249 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.402208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsv8v\" (UniqueName: \"kubernetes.io/projected/6750ba0a-2d11-4cff-a836-ff02a315a8f9-kube-api-access-gsv8v\") pod \"perf-node-gather-daemonset-fbgl5\" (UID: \"6750ba0a-2d11-4cff-a836-ff02a315a8f9\") " pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.496384 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.496318 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:31.546849 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.546821 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dc4b576ff-gmjwn_853b346b-9b40-42d6-9fbf-aa2c98c28fd7/console/0.log" Jul 18 03:24:31.633962 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:31.633942 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5"] Jul 18 03:24:31.635597 ip-10-0-138-139 kubenswrapper[2560]: W0718 03:24:31.635565 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6750ba0a_2d11_4cff_a836_ff02a315a8f9.slice/crio-6a8486c72fe01ad2d51558285759dc25fb986af66060e1ee46b8e17cf2f7da6c WatchSource:0}: Error finding container 6a8486c72fe01ad2d51558285759dc25fb986af66060e1ee46b8e17cf2f7da6c: Status 404 returned error can't find the container with id 6a8486c72fe01ad2d51558285759dc25fb986af66060e1ee46b8e17cf2f7da6c Jul 18 03:24:32.217944 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:32.217903 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" event={"ID":"6750ba0a-2d11-4cff-a836-ff02a315a8f9","Type":"ContainerStarted","Data":"ffdd6f4eab860039be778c2d9b7121ec9f74991b19e31da8e42eb2bee82044fa"} Jul 18 03:24:32.218392 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:32.217964 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" event={"ID":"6750ba0a-2d11-4cff-a836-ff02a315a8f9","Type":"ContainerStarted","Data":"6a8486c72fe01ad2d51558285759dc25fb986af66060e1ee46b8e17cf2f7da6c"} Jul 18 03:24:32.218392 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:32.218044 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:32.233978 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:32.233930 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" podStartSLOduration=1.233917211 podStartE2EDuration="1.233917211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 03:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 03:24:32.233337843 +0000 UTC m=+2664.022111616" watchObservedRunningTime="2026-07-18 03:24:32.233917211 +0000 UTC m=+2664.022690980" Jul 18 03:24:33.129568 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:33.129538 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5nt6c_e317b0e3-93bf-4318-a2ad-8d683762e301/dns/0.log" Jul 18 03:24:33.145352 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:33.145327 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5nt6c_e317b0e3-93bf-4318-a2ad-8d683762e301/kube-rbac-proxy/0.log" Jul 18 03:24:33.245947 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:33.245918 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7zpc4_fd825113-9c7b-4f2e-be4d-7d4074bc1c65/dns-node-resolver/0.log" Jul 18 03:24:33.801678 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:33.801647 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b9fb5fd8d-pmrlx_8448ca3a-17a8-43dc-afc7-4931b4f83318/registry/0.log" Jul 18 03:24:33.863112 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:33.863084 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9b5gs_a7793953-fe81-4ba2-af53-15518dc880e4/node-ca/0.log" Jul 18 03:24:34.765131 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:34.765103 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdswkkg_f55562f7-6c54-4e59-b9e7-0b27e71de895/istio-proxy/0.log" Jul 18 03:24:35.057701 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:35.057617 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/0.log" Jul 18 03:24:35.116682 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:35.116651 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/1.log" Jul 18 03:24:35.155994 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:35.155947 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xqsfl_8516c60e-702f-4312-8850-40dac347bfe5/payload-pre-processing/0.log" Jul 18 03:24:35.193544 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:35.193511 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vzgtk_f30942d9-047a-4f67-91e3-f229658a28ea/payload-processing/0.log" Jul 18 03:24:35.878656 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:35.878620 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9dw4n_a38f5409-3fa9-4a9e-9ada-4371a96c516e/serve-healthcheck-canary/0.log" Jul 18 03:24:36.514182 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:36.514151 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wbsjv_02c7cf7d-935f-4bcc-8905-bd040c739002/kube-rbac-proxy/0.log" Jul 18 03:24:36.531846 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:36.531815 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wbsjv_02c7cf7d-935f-4bcc-8905-bd040c739002/exporter/0.log" Jul 18 03:24:36.550592 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:36.550563 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wbsjv_02c7cf7d-935f-4bcc-8905-bd040c739002/extractor/0.log" Jul 18 03:24:38.233061 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:38.233034 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tn4zf/perf-node-gather-daemonset-fbgl5" Jul 18 03:24:38.597909 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:38.597888 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hskm5_748f4be0-e25a-4abb-96b1-44599411a0a3/manager/0.log" Jul 18 03:24:39.125205 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:39.125153 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d6d58f489-m6grn_1284e775-8b46-4e7d-aa84-3b59ebaa1de8/manager/0.log" Jul 18 03:24:39.144854 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:39.144828 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5nzlm_6591a5a9-0b8f-45c2-9f7a-46169e6bc7ea/server/0.log" Jul 18 03:24:39.476205 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:39.476110 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f9896c96c-f4klz_4bfd634f-8e3b-4236-9659-bdd5a2ea9655/manager/0.log" Jul 18 03:24:39.490499 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:39.490476 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f9896c96c-f4klz_4bfd634f-8e3b-4236-9659-bdd5a2ea9655/copy-manifests/0.log" Jul 18 03:24:40.871736 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:40.871706 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59945b78f9-42rj8_39800342-6207-473f-a2ba-06b88d388c5a/manager/0.log" Jul 18 03:24:40.917952 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:40.917928 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-x77hj_0993167e-3b5d-4a6b-ae4d-a970e352487e/openshift-lws-operator/0.log" Jul 18 03:24:41.593157 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:41.593128 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdswkkg_f55562f7-6c54-4e59-b9e7-0b27e71de895/istio-proxy/0.log" Jul 18 03:24:41.854225 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:41.854127 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/0.log" Jul 18 03:24:41.915920 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:41.915876 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/1.log" Jul 18 03:24:41.942198 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:41.942153 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xqsfl_8516c60e-702f-4312-8850-40dac347bfe5/payload-pre-processing/0.log" Jul 18 03:24:41.970834 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:41.970805 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vzgtk_f30942d9-047a-4f67-91e3-f229658a28ea/payload-processing/0.log" Jul 18 03:24:43.264944 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:43.264886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdswkkg_f55562f7-6c54-4e59-b9e7-0b27e71de895/istio-proxy/0.log" Jul 18 03:24:43.535305 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:43.535193 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/0.log" Jul 18 03:24:43.597761 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:43.597733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-th65c_1db93015-6201-403c-a36d-d5d76807ee72/istio-proxy/1.log" Jul 18 03:24:43.626585 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:43.626556 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xqsfl_8516c60e-702f-4312-8850-40dac347bfe5/payload-pre-processing/0.log" Jul 18 03:24:43.660984 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:43.660955 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vzgtk_f30942d9-047a-4f67-91e3-f229658a28ea/payload-processing/0.log" Jul 18 03:24:47.423033 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.423005 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5sj4d_06678ee4-c1e6-4e29-8c7f-eff88e49a516/kube-multus/0.log" Jul 18 03:24:47.615295 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.615268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/kube-multus-additional-cni-plugins/0.log" Jul 18 03:24:47.633521 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.633496 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/egress-router-binary-copy/0.log" Jul 18 03:24:47.651230 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.651209 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/cni-plugins/0.log" Jul 18 03:24:47.670260 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.670239 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/bond-cni-plugin/0.log" Jul 18 03:24:47.688114 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.688059 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/routeoverride-cni/0.log" Jul 18 03:24:47.708496 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.708478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/whereabouts-cni-bincopy/0.log" Jul 18 03:24:47.725873 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.725850 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x5g6r_c9668b8d-42c4-435f-a8a6-619519fd4f93/whereabouts-cni/0.log" Jul 18 03:24:47.963642 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.963530 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jzqvv_e42b787c-d4c0-4bf0-bf67-bc10adb618ff/network-metrics-daemon/0.log" Jul 18 03:24:47.980423 ip-10-0-138-139 kubenswrapper[2560]: I0718 03:24:47.980395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jzqvv_e42b787c-d4c0-4bf0-bf67-bc10adb618ff/kube-rbac-proxy/0.log"