Jul 08 11:59:51.665248 ip-10-0-138-160 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 11:59:51.665261 ip-10-0-138-160 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 11:59:51.665269 ip-10-0-138-160 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 11:59:51.665742 ip-10-0-138-160 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 12:00:01.702918 ip-10-0-138-160 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 12:00:01.702941 ip-10-0-138-160 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 44eeff68bcdf4c2697df7902034e08a4 -- Jul 08 12:02:07.198288 ip-10-0-138-160 systemd[1]: Starting Kubernetes Kubelet... Jul 08 12:02:07.592788 ip-10-0-138-160 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 12:02:07.592788 ip-10-0-138-160 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 12:02:07.592788 ip-10-0-138-160 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 12:02:07.592788 ip-10-0-138-160 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 12:02:07.592788 ip-10-0-138-160 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 12:02:07.593715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.593619 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 12:02:07.599137 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599106 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 12:02:07.599137 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599134 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 12:02:07.599137 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599137 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 12:02:07.599137 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599141 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 12:02:07.599137 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599144 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 12:02:07.599137 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599147 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599150 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599154 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599156 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599159 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599161 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599164 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599167 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599169 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599172 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599175 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599180 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599185 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599189 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599192 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599194 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599197 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599199 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599203 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 12:02:07.599375 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599205 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599208 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599211 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599213 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599216 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599218 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599221 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599224 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599226 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599230 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599233 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599235 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599237 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599240 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599243 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599247 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599250 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599252 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599255 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599257 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 12:02:07.599868 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599260 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599263 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599266 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599268 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599271 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599274 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599276 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599279 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599282 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599286 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599291 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599294 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599297 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599299 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599303 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599305 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599308 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599311 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599314 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599317 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 12:02:07.600502 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599319 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599322 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599325 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599327 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599330 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599333 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599335 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599339 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599342 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599345 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599348 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599351 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599353 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599357 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599360 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599364 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599367 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599370 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599373 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 12:02:07.601034 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599377 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599380 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599382 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599385 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599388 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599391 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599393 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599396 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599398 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599401 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599403 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599406 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599409 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599411 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599414 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.599416 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600211 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600221 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600224 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600227 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 12:02:07.601503 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600231 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600234 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600237 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600240 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600242 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600245 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600248 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600250 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600253 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600256 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600258 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600261 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600265 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600271 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600274 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600278 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600282 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600285 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600288 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 12:02:07.602046 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600291 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600294 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600297 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600300 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600305 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600308 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600311 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600314 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600316 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600319 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600322 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600324 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600327 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600331 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600333 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600336 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600339 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600342 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600344 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 12:02:07.602564 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600347 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600350 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600352 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600355 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600357 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600360 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600362 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600365 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600368 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600370 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600373 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600376 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600379 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600381 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600384 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600386 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600389 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600391 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600394 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600397 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 12:02:07.603050 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600399 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600402 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600405 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600407 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600410 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600413 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600416 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600419 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600421 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600424 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600427 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600429 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600432 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600434 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600437 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600439 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600442 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600447 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600449 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 12:02:07.603544 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600452 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600454 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600458 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600461 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600463 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600466 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600469 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600471 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600474 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600476 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600479 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600481 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600484 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600487 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600511 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600515 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600518 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600522 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600984 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600993 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 12:02:07.604047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600996 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.600999 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601003 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601008 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601011 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601014 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601017 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601020 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601023 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601025 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601028 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601031 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601034 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601037 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601040 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601044 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601046 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601049 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601052 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 12:02:07.604560 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601055 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601058 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601060 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601063 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601065 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601067 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601070 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601072 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601075 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601078 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601080 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601082 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601087 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601090 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601092 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601095 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601097 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601100 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601102 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601106 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 12:02:07.605056 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601108 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601111 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601114 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601116 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601119 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601122 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601124 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601127 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601131 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601133 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601136 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601139 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601141 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601143 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601146 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601148 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601153 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601156 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601159 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601161 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 12:02:07.605545 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601164 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601166 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601168 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601171 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601175 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601178 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601180 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601183 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601186 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601189 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601191 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601194 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601196 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601199 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601201 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601204 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601206 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601212 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601216 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601218 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 12:02:07.606067 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601221 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601225 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601228 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601231 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601233 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601236 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601239 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601241 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601246 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601250 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601253 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601256 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601258 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601261 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601264 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601266 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601271 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.601273 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601355 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601364 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601372 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 12:02:07.606546 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601379 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601386 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601391 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601398 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601405 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601408 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601411 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601415 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601419 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601422 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601425 2560 flags.go:64] FLAG: --cgroup-root="" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601428 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601432 2560 flags.go:64] FLAG: --client-ca-file="" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601437 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601440 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601446 2560 flags.go:64] FLAG: --cluster-domain="" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601448 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601452 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601455 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601460 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601464 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601467 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601470 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 12:02:07.607075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601474 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601477 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601480 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601483 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601488 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601491 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601496 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601499 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601502 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601505 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601508 2560 flags.go:64] FLAG: --enable-server="true" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601512 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601517 2560 flags.go:64] FLAG: --event-burst="100" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601520 2560 flags.go:64] FLAG: --event-qps="50" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601523 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601527 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601530 2560 flags.go:64] FLAG: --eviction-hard="" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601534 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601536 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601539 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601542 2560 flags.go:64] FLAG: --eviction-soft="" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601546 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601549 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601552 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601555 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 12:02:07.607619 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601558 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601561 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601564 2560 flags.go:64] FLAG: --feature-gates="" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601569 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601572 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601575 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601579 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601583 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601586 2560 flags.go:64] FLAG: --help="false" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601589 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601592 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601596 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601600 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601603 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601607 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601610 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601613 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601616 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601619 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601622 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601625 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601628 2560 flags.go:64] FLAG: --kube-reserved="" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601631 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601634 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 12:02:07.608305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601637 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601640 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601642 2560 flags.go:64] FLAG: --lock-file="" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601645 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601648 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601651 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601657 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601660 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601663 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601666 2560 flags.go:64] FLAG: --logging-format="text" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601669 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601674 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601676 2560 flags.go:64] FLAG: --manifest-url="" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601679 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601684 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601687 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601692 2560 flags.go:64] FLAG: --max-pods="110" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601695 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601698 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601702 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601705 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601708 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601711 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601714 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601723 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 12:02:07.608917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601726 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601729 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601732 2560 flags.go:64] FLAG: --pod-cidr="" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601735 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601741 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601744 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601747 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601750 2560 flags.go:64] FLAG: --port="10250" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601753 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601756 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-097f16c8c188f2eca" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601759 2560 flags.go:64] FLAG: --qos-reserved="" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601762 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601765 2560 flags.go:64] FLAG: --register-node="true" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601768 2560 flags.go:64] FLAG: --register-with-taints="" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601772 2560 flags.go:64] FLAG: --registry-burst="10" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601776 2560 flags.go:64] FLAG: --registry-qps="5" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601779 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601782 2560 flags.go:64] FLAG: --reserved-memory="" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601788 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601791 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601795 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601798 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601801 2560 flags.go:64] FLAG: --runonce="false" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601804 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601807 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 12:02:07.609618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601810 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601813 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601818 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601821 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601824 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601827 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601830 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601834 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601837 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601840 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601843 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601846 2560 flags.go:64] FLAG: --system-cgroups="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601849 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601855 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601858 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601861 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601866 2560 flags.go:64] FLAG: --tls-min-version="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601869 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601872 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601875 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601878 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601881 2560 flags.go:64] FLAG: --v="2" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601904 2560 flags.go:64] FLAG: --version="false" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601916 2560 flags.go:64] FLAG: --vmodule="" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601921 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 12:02:07.610394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.601926 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602053 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602058 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602061 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602065 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602068 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602071 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602073 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602076 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602082 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602084 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602087 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602090 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602092 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602095 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602098 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602100 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602103 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602106 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602108 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 12:02:07.611035 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602111 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602113 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602116 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602119 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602122 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602124 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602127 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602129 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602132 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602134 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602137 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602140 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602144 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602147 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602151 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602154 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602157 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602159 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602162 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602165 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 12:02:07.611517 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602167 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602171 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602174 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602177 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602180 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602182 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602185 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602187 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602190 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602193 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602196 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602198 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602201 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602203 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602206 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602209 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602211 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602214 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602216 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602219 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 12:02:07.612064 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602221 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602224 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602226 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602229 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602233 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602236 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602238 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602240 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602243 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602246 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602248 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602251 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602255 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602259 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602262 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602265 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602268 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602270 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602273 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 12:02:07.612552 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602276 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602279 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602281 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602284 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602286 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602289 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602292 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602294 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602297 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602300 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602302 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602305 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602307 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602310 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602313 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602317 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602320 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602323 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602327 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 12:02:07.613047 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602330 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.602332 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.603040 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.611006 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.611032 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.611923 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.611931 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612003 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612010 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612014 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612017 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612021 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612023 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612026 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612029 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 12:02:07.613524 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612032 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612035 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612038 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612040 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612043 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612046 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612048 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612051 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612053 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612056 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612058 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612061 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612064 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612066 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612070 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612072 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612075 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612078 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612081 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 12:02:07.613976 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612084 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612087 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612096 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612099 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612102 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612104 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612107 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612109 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612112 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612115 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612117 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612120 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612123 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612125 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612128 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612130 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612133 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612136 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612138 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612141 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 12:02:07.614447 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612143 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612146 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612148 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612151 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612153 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612156 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612158 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612161 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612164 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612166 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612169 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612173 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612178 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612181 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612185 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612193 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612196 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612199 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612201 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 12:02:07.614955 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612204 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612207 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612209 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612212 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612215 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612218 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612220 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612223 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612225 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612228 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612233 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612237 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612239 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612242 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612245 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612248 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612251 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612254 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612257 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 12:02:07.615419 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612260 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612262 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612265 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612267 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612270 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612273 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612276 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612279 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612282 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612284 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612293 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612297 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612301 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612304 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.612310 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 12:02:07.615958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612463 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612468 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612471 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612474 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612476 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612479 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612482 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612485 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612487 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612490 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612498 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612501 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612504 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612506 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612509 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612511 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612514 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612518 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612520 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612523 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 12:02:07.616347 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612525 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612528 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612530 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612533 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612536 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612539 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612541 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612544 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612553 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612555 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612559 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612562 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612564 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612567 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612570 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612572 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612575 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612578 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612580 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612582 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 12:02:07.616829 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612585 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612587 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612590 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612592 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612595 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612597 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612600 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612602 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612605 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612608 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612610 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612613 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612616 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612618 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612621 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612624 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612627 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612630 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612633 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612635 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 12:02:07.617378 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612639 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612650 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612653 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612656 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612660 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612663 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612666 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612669 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612672 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612675 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612677 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612680 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612682 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612685 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612688 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612691 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612695 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612697 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612700 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 12:02:07.617964 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612702 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612705 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612708 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612710 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612713 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612716 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612718 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612721 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612723 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612726 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612729 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612731 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612734 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612737 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612739 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612748 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612751 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612753 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612756 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 12:02:07.618533 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:07.612758 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 12:02:07.619087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.612764 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 12:02:07.619087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.612939 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 12:02:07.619087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.615551 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 12:02:07.619087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.617197 2560 server.go:1019] "Starting client certificate rotation" Jul 08 12:02:07.619087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.617325 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 12:02:07.619087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.618043 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 12:02:07.620957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.620936 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 12:02:07.624271 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.624239 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 12:02:07.636662 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.636634 2560 log.go:25] "Validated CRI v1 runtime API" Jul 08 12:02:07.639387 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.639368 2560 log.go:25] "Validated CRI v1 image API" Jul 08 12:02:07.640424 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.640397 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 12:02:07.644487 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.644453 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7ad9a1d6-9977-4142-b517-8cbec0c20524:/dev/nvme0n1p3 fbd2c808-e0bf-430e-b269-33112d37ef10:/dev/nvme0n1p4] Jul 08 12:02:07.644487 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.644482 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 12:02:07.649872 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.649714 2560 manager.go:216] Machine: {Timestamp:2026-07-08 12:02:07.648386696 +0000 UTC m=+0.331439485 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111917 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec275065831211a56afcab145203b70f SystemUUID:ec275065-8312-11a5-6afc-ab145203b70f BootID:44eeff68-bcdf-4c26-97df-7902034e08a4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:1e:64:c2:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:1e:64:c2:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:95:6a:cc:cd:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 12:02:07.649872 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.649847 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 12:02:07.650132 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.650003 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 12:02:07.650806 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.650788 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 12:02:07.653568 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.653526 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 12:02:07.653720 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.653571 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 12:02:07.655105 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.655090 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 12:02:07.655161 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.655107 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 12:02:07.655161 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.655116 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 12:02:07.655898 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.655871 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 12:02:07.657612 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.657597 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 12:02:07.662155 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662135 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 12:02:07.662308 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662297 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 12:02:07.662351 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662334 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 08 12:02:07.662386 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662356 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 12:02:07.662386 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662373 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 12:02:07.662386 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662385 2560 kubelet.go:398] "Adding apiserver pod source" Jul 08 12:02:07.662503 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662395 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 12:02:07.662853 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662840 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 12:02:07.662907 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.662865 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 12:02:07.665784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.665763 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 12:02:07.668028 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.668010 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 12:02:07.668126 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.668043 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 12:02:07.669964 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.669943 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.669971 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.669982 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.669993 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670004 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670011 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670020 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670031 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670039 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 12:02:07.670048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670045 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 12:02:07.670409 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670068 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 12:02:07.670409 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670075 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 12:02:07.670876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.670856 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 12:02:07.671023 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.671013 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 12:02:07.674509 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.674477 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 12:02:07.674509 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.674480 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 12:02:07.674824 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.674811 2560 server.go:1268] "Started kubelet" Jul 08 12:02:07.675024 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.674961 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 12:02:07.675124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.675035 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 12:02:07.675195 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.675048 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 12:02:07.675823 ip-10-0-138-160 systemd[1]: Started Kubernetes Kubelet. Jul 08 12:02:07.677203 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.677175 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 12:02:07.677305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.677252 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 08 12:02:07.680799 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.680761 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwj6z" Jul 08 12:02:07.682281 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.682249 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 12:02:07.685588 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.685564 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 12:02:07.685732 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.685575 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 12:02:07.685838 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.685716 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:07.686404 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.686382 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 12:02:07.686768 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.686569 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 12:02:07.686768 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.686589 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 12:02:07.686768 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.686630 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 12:02:07.686981 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.686784 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 08 12:02:07.686981 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.686794 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 08 12:02:07.687153 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687136 2560 factory.go:55] Registering systemd factory Jul 08 12:02:07.687210 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687156 2560 factory.go:223] Registration of the systemd container factory successfully Jul 08 12:02:07.687453 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687409 2560 factory.go:153] Registering CRI-O factory Jul 08 12:02:07.687453 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687427 2560 factory.go:223] Registration of the crio container factory successfully Jul 08 12:02:07.687599 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687544 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 12:02:07.687599 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687564 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwj6z" Jul 08 12:02:07.687599 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687581 2560 factory.go:103] Registering Raw factory Jul 08 12:02:07.687716 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.686588 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-160.ec2.internal.18c04ed4a6a7d3bc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-160.ec2.internal,UID:ip-10-0-138-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-160.ec2.internal,},FirstTimestamp:2026-07-08 12:02:07.674766268 +0000 UTC m=+0.357819057,LastTimestamp:2026-07-08 12:02:07.674766268 +0000 UTC m=+0.357819057,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-160.ec2.internal,}" Jul 08 12:02:07.687716 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.687633 2560 manager.go:1196] Started watching for new ooms in manager Jul 08 12:02:07.688425 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.688309 2560 manager.go:349] Starting recovery of all containers Jul 08 12:02:07.697535 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.697329 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 12:02:07.700518 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.700478 2560 manager.go:354] Recovery completed Jul 08 12:02:07.701422 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.701398 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-160.ec2.internal\" not found" node="ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.706165 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.706148 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.708545 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.708523 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.708661 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.708559 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.708661 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.708574 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:07.709182 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.709164 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 12:02:07.709182 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.709182 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 12:02:07.709322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.709205 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 12:02:07.709322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.709243 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.710010 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.709970 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.710010 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.710010 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.710105 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.710023 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:07.711411 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.711395 2560 policy_none.go:49] "None policy: Start" Jul 08 12:02:07.711486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.711419 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 12:02:07.711486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.711433 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 12:02:07.713594 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.713578 2560 policy_none.go:47] "Start" Jul 08 12:02:07.754671 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.754644 2560 manager.go:337] "Starting Device Plugin manager" Jul 08 12:02:07.754811 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.754687 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 12:02:07.754811 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.754700 2560 server.go:87] "Starting device plugin registration server" Jul 08 12:02:07.755060 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.755047 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 12:02:07.755099 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.755064 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 12:02:07.755246 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.755219 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 12:02:07.755355 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.755319 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 12:02:07.755355 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.755329 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 12:02:07.756068 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.756045 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 12:02:07.756186 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.756104 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:07.855269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.855161 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.858474 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.858450 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.858594 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.858492 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.858594 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.858506 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:07.858594 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.858537 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.867173 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.867149 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.867325 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.867180 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-160.ec2.internal\": node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:07.878629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.878592 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 12:02:07.880260 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.880236 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 12:02:07.880384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.880295 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 12:02:07.880384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.880329 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 12:02:07.880384 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.880371 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 12:02:07.881429 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.881409 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:07.882762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.882738 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 12:02:07.980672 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.980610 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"] Jul 08 12:02:07.980876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.980713 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.982220 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:07.982202 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:07.982706 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.982689 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.982790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.982718 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.982790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.982730 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:07.985176 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.985158 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.985278 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.985261 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.985322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.985295 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.986020 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.986005 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.986082 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.986038 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.986082 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.986051 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:07.986082 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.986053 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.986082 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.986072 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.986082 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.986080 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:07.988143 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.988121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbb64130c7b615dfb98b4e6af62c0947-config\") pod \"kube-apiserver-proxy-ip-10-0-138-160.ec2.internal\" (UID: \"bbb64130c7b615dfb98b4e6af62c0947\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.988235 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.988158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6d9a6fed735c32fc3e333fe15459579-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d6d9a6fed735c32fc3e333fe15459579\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.988235 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.988187 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6d9a6fed735c32fc3e333fe15459579-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d6d9a6fed735c32fc3e333fe15459579\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.988452 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.988434 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:07.988491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.988482 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 12:02:07.989332 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.989315 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory" Jul 08 12:02:07.989418 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.989351 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 12:02:07.989418 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:07.989366 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID" Jul 08 12:02:08.015142 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.015110 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-160.ec2.internal\" not found" node="ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.019943 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.019923 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-160.ec2.internal\" not found" node="ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.083202 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.083164 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.088852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.088823 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbb64130c7b615dfb98b4e6af62c0947-config\") pod \"kube-apiserver-proxy-ip-10-0-138-160.ec2.internal\" (UID: \"bbb64130c7b615dfb98b4e6af62c0947\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.088946 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.088859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6d9a6fed735c32fc3e333fe15459579-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d6d9a6fed735c32fc3e333fe15459579\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.088946 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.088878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6d9a6fed735c32fc3e333fe15459579-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d6d9a6fed735c32fc3e333fe15459579\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.089055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.088959 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6d9a6fed735c32fc3e333fe15459579-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d6d9a6fed735c32fc3e333fe15459579\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.089055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.088984 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6d9a6fed735c32fc3e333fe15459579-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d6d9a6fed735c32fc3e333fe15459579\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.089055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.088964 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbb64130c7b615dfb98b4e6af62c0947-config\") pod \"kube-apiserver-proxy-ip-10-0-138-160.ec2.internal\" (UID: \"bbb64130c7b615dfb98b4e6af62c0947\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.184090 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.184006 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.284665 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.284617 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.319154 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.319114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.324654 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.324565 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:08.385098 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.385050 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.485663 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.485583 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.587791 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.587749 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.617296 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.617258 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 12:02:08.617916 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.617459 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 12:02:08.617916 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.617491 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 12:02:08.687895 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.687869 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.692393 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.692337 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 11:57:07 +0000 UTC" deadline="2028-01-06 00:50:02.913451461 +0000 UTC" Jul 08 12:02:08.692393 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:08.692391 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13116h47m54.221064823s" Jul 08 12:02:08.789058 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.788952 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.889834 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.889803 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:08.990965 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:08.990916 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:09.015545 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.015509 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 12:02:09.091673 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.091637 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:09.192584 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.192550 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found" Jul 08 12:02:09.203537 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:09.203485 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb64130c7b615dfb98b4e6af62c0947.slice/crio-0b494aa83b5079080ead305bb7d2ca125c106568f48bd31f934fb43774c666ab WatchSource:0}: Error finding container 0b494aa83b5079080ead305bb7d2ca125c106568f48bd31f934fb43774c666ab: Status 404 returned error can't find the container with id 0b494aa83b5079080ead305bb7d2ca125c106568f48bd31f934fb43774c666ab Jul 08 12:02:09.203817 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:09.203793 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d9a6fed735c32fc3e333fe15459579.slice/crio-72ec879af458768031e108ed0d50a0e3b3321e93c0139e69873e4127dbfa1c94 WatchSource:0}: Error finding container 72ec879af458768031e108ed0d50a0e3b3321e93c0139e69873e4127dbfa1c94: Status 404 returned error can't find the container with id 72ec879af458768031e108ed0d50a0e3b3321e93c0139e69873e4127dbfa1c94 Jul 08 12:02:09.209845 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.209819 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 12:02:09.247246 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.247211 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 12:02:09.287508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.287463 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" Jul 08 12:02:09.300536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.300445 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 12:02:09.301613 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.301594 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" Jul 08 12:02:09.315280 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.315253 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 12:02:09.666336 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.666221 2560 apiserver.go:52] "Watching apiserver" Jul 08 12:02:09.676287 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.676253 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 12:02:09.677521 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.677493 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v","openshift-cluster-node-tuning-operator/tuned-9tdgl","openshift-image-registry/node-ca-dmgs9","openshift-multus/multus-additional-cni-plugins-tmvlb","openshift-multus/multus-c4brj","openshift-ovn-kubernetes/ovnkube-node-xkcwp","kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal","openshift-dns/node-resolver-xgt62","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal","openshift-multus/network-metrics-daemon-f8lqw","openshift-network-diagnostics/network-check-target-rnpfb","openshift-network-operator/iptables-alerter-tfjj8","kube-system/global-pull-secret-syncer-rc5xg","kube-system/konnectivity-agent-ndns4"] Jul 08 12:02:09.681787 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.681762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:09.681870 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.681847 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:09.684039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.684014 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.684112 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.684092 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:09.686375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.686357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.687365 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.687330 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 12:02:09.688808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.688785 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.689160 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.689134 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.689284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.689168 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.689387 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.689371 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 12:02:09.689473 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.689459 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vst85\"" Jul 08 12:02:09.691304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.691285 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.691410 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.691362 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 12:02:09.691410 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.691398 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.692494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.692476 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.692595 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.692513 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 12:02:09.692755 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.692726 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-58tkw\"" Jul 08 12:02:09.692844 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.692827 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 12:02:09.693851 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.693827 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.693987 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.693965 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 12:02:09.694097 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.694069 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w42pw\"" Jul 08 12:02:09.696418 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.696394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.696559 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.696522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-system-cni-dir\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.697280 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.696557 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cnibin\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.697280 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697016 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 12:02:09.697280 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.697613 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:09.697833 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697809 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.697919 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697831 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-host\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.697919 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697884 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.698024 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84svc\" (UniqueName: \"kubernetes.io/projected/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-kube-api-access-84svc\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.698024 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.697985 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-kubelet-config\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.698095 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698035 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 12:02:09.698229 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698208 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 12:02:09.698265 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698023 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cni-binary-copy\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.698299 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.698345 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-os-release\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.698392 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.698446 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698414 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8jw\" (UniqueName: \"kubernetes.io/projected/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-kube-api-access-6q8jw\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:09.698491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698449 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-dbus\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.698491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-serviceca\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.699068 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698751 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lfkvw\"" Jul 08 12:02:09.699068 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qx5l\" (UniqueName: \"kubernetes.io/projected/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-kube-api-access-4qx5l\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.699068 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.698994 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 12:02:09.699276 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.699218 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.699557 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.699536 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 12:02:09.700822 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.700157 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.700822 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.700220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 12:02:09.700822 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.700695 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-scm6v\"" Jul 08 12:02:09.703152 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.703125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.703442 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.703354 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.703642 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.703625 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.703806 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.703790 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2hgbt\"" Jul 08 12:02:09.705769 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.705750 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.706340 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.706307 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 12:02:09.706340 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.706333 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.706540 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.706524 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.706617 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.706573 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lkkn5\"" Jul 08 12:02:09.708095 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.708079 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.708326 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.708303 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:09.708419 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.708395 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:09.709719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.709699 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.709834 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.709760 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 12:02:09.709834 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.709828 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p98xd\"" Jul 08 12:02:09.711060 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.711014 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.713499 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.713479 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 12:02:09.714066 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.714042 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 12:02:09.714066 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.714063 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bn55m\"" Jul 08 12:02:09.714223 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.714203 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 12:02:09.728943 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.728912 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mtfjs" Jul 08 12:02:09.739037 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.739015 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mtfjs" Jul 08 12:02:09.787777 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.787746 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 12:02:09.799069 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-cnibin\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.799245 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-kubelet\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.799245 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799245 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-var-lib-kubelet\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.799245 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799216 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-host\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-os-release\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799277 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-node-log\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-run-netns\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799330 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-system-cni-dir\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-systemd-units\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-cni-bin\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799395 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-os-release\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799407 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-system-cni-dir\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-socket-dir-parent\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.799462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5adcca2c-870d-4702-907c-a9023968bdbc-host-slash\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-host\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-lib-modules\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799549 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-host\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799564 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86v97\" (UniqueName: \"kubernetes.io/projected/5adcca2c-870d-4702-907c-a9023968bdbc-kube-api-access-86v97\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799620 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cni-binary-copy\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-var-lib-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799718 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8jw\" (UniqueName: \"kubernetes.io/projected/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-kube-api-access-6q8jw\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-dbus\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799808 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovnkube-config\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-sys\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.799868 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799876 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-socket-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.799934 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.799943 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-registration-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.799986 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:10.299941304 +0000 UTC m=+2.982994109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800023 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-dbus\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-device-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800118 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-cni-netd\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-kubernetes\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-system-cni-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-etc-selinux\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlsrd\" (UniqueName: \"kubernetes.io/projected/9ed760d9-54eb-4b77-aef5-a9133d175fd3-kube-api-access-rlsrd\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovnkube-script-lib\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-hosts-file\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-tmp-dir\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800381 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cni-binary-copy\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800414 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-systemd\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-os-release\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.800816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800478 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-etc-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800508 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-log-socket\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800586 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysconfig\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-cni-bin\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-tuned\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800680 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-slash\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.800752 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-env-overrides\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.800826 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:10.300802636 +0000 UTC m=+2.983855415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovn-node-metrics-cert\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-cni-binary-copy\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.800970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5adcca2c-870d-4702-907c-a9023968bdbc-iptables-alerter-script\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801019 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-kubelet-config\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb3c4f96-1c10-47f9-b03c-76bebeef9fa4-konnectivity-ca\") pod \"konnectivity-agent-ndns4\" (UID: \"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4\") " pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.801615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801087 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-modprobe-d\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysctl-d\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd4pl\" (UniqueName: \"kubernetes.io/projected/62dba4ee-bfa1-4812-8000-17d4e4377bd2-kube-api-access-qd4pl\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-cni-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801582 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-k8s-cni-cncf-io\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysctl-conf\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62dba4ee-bfa1-4812-8000-17d4e4377bd2-tmp\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801123 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-kubelet-config\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801678 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-cni-multus\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-etc-kubernetes\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801700 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801750 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8w2w\" (UniqueName: \"kubernetes.io/projected/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-kube-api-access-c8w2w\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-sys-fs\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-ovn\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.801852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802062 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-kubelet\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.802432 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb3c4f96-1c10-47f9-b03c-76bebeef9fa4-agent-certs\") pod \"konnectivity-agent-ndns4\" (UID: \"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4\") " pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802300 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-run\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-netns\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-hostroot\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-conf-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802530 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-serviceca\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.803269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.802624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qx5l\" (UniqueName: \"kubernetes.io/projected/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-kube-api-access-4qx5l\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804108 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-serviceca\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804471 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cnibin\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-daemon-config\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-multus-certs\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdcn7\" (UniqueName: \"kubernetes.io/projected/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-kube-api-access-mdcn7\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.804714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84svc\" (UniqueName: \"kubernetes.io/projected/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-kube-api-access-84svc\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.805150 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804727 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-cnibin\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.805150 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-systemd\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.805150 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.804945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vcs4\" (UniqueName: \"kubernetes.io/projected/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-kube-api-access-8vcs4\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.805609 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.805585 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.808342 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.808317 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 12:02:09.811590 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.811569 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8jw\" (UniqueName: \"kubernetes.io/projected/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-kube-api-access-6q8jw\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:09.814129 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.814101 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qx5l\" (UniqueName: \"kubernetes.io/projected/0f1897f6-14e3-4e50-b3a6-6c15c91ae66c-kube-api-access-4qx5l\") pod \"node-ca-dmgs9\" (UID: \"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c\") " pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:09.814486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.814463 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84svc\" (UniqueName: \"kubernetes.io/projected/7369164f-0ab7-4a79-ab87-1d7a5c77ef21-kube-api-access-84svc\") pod \"multus-additional-cni-plugins-tmvlb\" (UID: \"7369164f-0ab7-4a79-ab87-1d7a5c77ef21\") " pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:09.890805 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.890746 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" event={"ID":"d6d9a6fed735c32fc3e333fe15459579","Type":"ContainerStarted","Data":"72ec879af458768031e108ed0d50a0e3b3321e93c0139e69873e4127dbfa1c94"} Jul 08 12:02:09.892018 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.891984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" event={"ID":"bbb64130c7b615dfb98b4e6af62c0947","Type":"ContainerStarted","Data":"0b494aa83b5079080ead305bb7d2ca125c106568f48bd31f934fb43774c666ab"} Jul 08 12:02:09.906210 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906179 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-systemd\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.906361 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-os-release\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.906361 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906236 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-etc-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906361 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-log-socket\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906361 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysconfig\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.906361 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906344 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysconfig\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.906361 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906345 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-systemd\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906359 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-log-socket\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-cni-bin\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906385 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-etc-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906385 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-cni-bin\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906394 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-os-release\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906405 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-tuned\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-slash\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-env-overrides\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovn-node-metrics-cert\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906547 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-slash\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906555 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-cni-binary-copy\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5adcca2c-870d-4702-907c-a9023968bdbc-iptables-alerter-script\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb3c4f96-1c10-47f9-b03c-76bebeef9fa4-konnectivity-ca\") pod \"konnectivity-agent-ndns4\" (UID: \"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4\") " pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906512 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.906629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-modprobe-d\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906666 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysctl-d\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd4pl\" (UniqueName: \"kubernetes.io/projected/62dba4ee-bfa1-4812-8000-17d4e4377bd2-kube-api-access-qd4pl\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-cni-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-k8s-cni-cncf-io\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysctl-conf\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906814 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62dba4ee-bfa1-4812-8000-17d4e4377bd2-tmp\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906813 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-modprobe-d\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906887 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-cni-multus\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907015 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysctl-d\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.906839 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-cni-multus\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907057 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-etc-kubernetes\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8w2w\" (UniqueName: \"kubernetes.io/projected/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-kube-api-access-c8w2w\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-sys-fs\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-ovn\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907153 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-env-overrides\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907199 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-kubelet\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907209 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-sys-fs\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.908608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907250 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-etc-kubernetes\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907253 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-ovn\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-kubelet\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907261 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-k8s-cni-cncf-io\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907297 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb3c4f96-1c10-47f9-b03c-76bebeef9fa4-konnectivity-ca\") pod \"konnectivity-agent-ndns4\" (UID: \"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4\") " pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb3c4f96-1c10-47f9-b03c-76bebeef9fa4-agent-certs\") pod \"konnectivity-agent-ndns4\" (UID: \"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4\") " pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-run\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-netns\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-hostroot\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907420 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-cni-binary-copy\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907414 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5adcca2c-870d-4702-907c-a9023968bdbc-iptables-alerter-script\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-conf-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907506 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-cni-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-daemon-config\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907492 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-netns\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907552 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-multus-certs\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907560 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-conf-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.909555 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdcn7\" (UniqueName: \"kubernetes.io/projected/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-kube-api-access-mdcn7\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907610 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-hostroot\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-systemd\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vcs4\" (UniqueName: \"kubernetes.io/projected/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-kube-api-access-8vcs4\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907661 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-run-multus-certs\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-cnibin\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-kubelet\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907725 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-run-systemd\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-var-lib-kubelet\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907777 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-cnibin\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-host\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907805 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-host-var-lib-kubelet\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907821 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-node-log\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-run-netns\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907865 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-var-lib-kubelet\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-systemd-units\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-cni-bin\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910200 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907909 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-node-log\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907892 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-socket-dir-parent\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907951 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-host\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5adcca2c-870d-4702-907c-a9023968bdbc-host-slash\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.907990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-systemd-units\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908013 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-cni-bin\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908044 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-socket-dir-parent\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908048 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-run-netns\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908063 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908064 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5adcca2c-870d-4702-907c-a9023968bdbc-host-slash\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-lib-modules\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86v97\" (UniqueName: \"kubernetes.io/projected/5adcca2c-870d-4702-907c-a9023968bdbc-kube-api-access-86v97\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908188 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-run\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-var-lib-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908223 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-multus-daemon-config\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.910841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908250 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-var-lib-openvswitch\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908264 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908282 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-lib-modules\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908283 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-sysctl-conf\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovnkube-config\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908315 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-sys\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-socket-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-registration-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-device-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-cni-netd\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-kubernetes\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908513 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-sys\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-system-cni-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908561 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-etc-selinux\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlsrd\" (UniqueName: \"kubernetes.io/projected/9ed760d9-54eb-4b77-aef5-a9133d175fd3-kube-api-access-rlsrd\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908610 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-device-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.911493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908614 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovnkube-script-lib\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-hosts-file\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-tmp-dir\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908909 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovnkube-config\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.908966 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-hosts-file\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909052 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-socket-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909160 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-kubernetes\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909211 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovnkube-script-lib\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909224 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-registration-dir\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909258 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ed760d9-54eb-4b77-aef5-a9133d175fd3-etc-selinux\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909306 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-system-cni-dir\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909154 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-host-cni-netd\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909491 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62dba4ee-bfa1-4812-8000-17d4e4377bd2-etc-tuned\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.909601 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-tmp-dir\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.910234 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62dba4ee-bfa1-4812-8000-17d4e4377bd2-tmp\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.910755 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb3c4f96-1c10-47f9-b03c-76bebeef9fa4-agent-certs\") pod \"konnectivity-agent-ndns4\" (UID: \"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4\") " pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:09.912055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.911543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-ovn-node-metrics-cert\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.915916 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.915877 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:09.915916 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.915919 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:09.916159 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.915930 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:09.916159 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:09.916010 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:10.415977729 +0000 UTC m=+3.099030530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:09.916659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.916597 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd4pl\" (UniqueName: \"kubernetes.io/projected/62dba4ee-bfa1-4812-8000-17d4e4377bd2-kube-api-access-qd4pl\") pod \"tuned-9tdgl\" (UID: \"62dba4ee-bfa1-4812-8000-17d4e4377bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:09.918583 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.918555 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86v97\" (UniqueName: \"kubernetes.io/projected/5adcca2c-870d-4702-907c-a9023968bdbc-kube-api-access-86v97\") pod \"iptables-alerter-tfjj8\" (UID: \"5adcca2c-870d-4702-907c-a9023968bdbc\") " pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:09.918659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.918570 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vcs4\" (UniqueName: \"kubernetes.io/projected/03e45ff0-4e75-4b46-bed1-2d447ad8afc1-kube-api-access-8vcs4\") pod \"ovnkube-node-xkcwp\" (UID: \"03e45ff0-4e75-4b46-bed1-2d447ad8afc1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:09.919070 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.919052 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdcn7\" (UniqueName: \"kubernetes.io/projected/453e570f-3d3f-4bdf-8475-dcf1b0bda30e-kube-api-access-mdcn7\") pod \"node-resolver-xgt62\" (UID: \"453e570f-3d3f-4bdf-8475-dcf1b0bda30e\") " pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:09.919194 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.919169 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8w2w\" (UniqueName: \"kubernetes.io/projected/d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4-kube-api-access-c8w2w\") pod \"multus-c4brj\" (UID: \"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4\") " pod="openshift-multus/multus-c4brj" Jul 08 12:02:09.919394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.919376 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlsrd\" (UniqueName: \"kubernetes.io/projected/9ed760d9-54eb-4b77-aef5-a9133d175fd3-kube-api-access-rlsrd\") pod \"aws-ebs-csi-driver-node-bc45v\" (UID: \"9ed760d9-54eb-4b77-aef5-a9133d175fd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:09.937335 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.937305 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 12:02:09.997324 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:09.997286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dmgs9" Jul 08 12:02:10.008248 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.008211 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" Jul 08 12:02:10.008978 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:10.008947 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1897f6_14e3_4e50_b3a6_6c15c91ae66c.slice/crio-ed243346d1be2ebb9581b71c1848ab53f572f30bb879fcb30c5eefef72cfa315 WatchSource:0}: Error finding container ed243346d1be2ebb9581b71c1848ab53f572f30bb879fcb30c5eefef72cfa315: Status 404 returned error can't find the container with id ed243346d1be2ebb9581b71c1848ab53f572f30bb879fcb30c5eefef72cfa315 Jul 08 12:02:10.016322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.016283 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c4brj" Jul 08 12:02:10.022473 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.022417 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:10.025497 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:10.025454 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7369164f_0ab7_4a79_ab87_1d7a5c77ef21.slice/crio-f161022e3105d0871e74a1be84cac28775e9a7eec44e9a6f7495a412da66dff6 WatchSource:0}: Error finding container f161022e3105d0871e74a1be84cac28775e9a7eec44e9a6f7495a412da66dff6: Status 404 returned error can't find the container with id f161022e3105d0871e74a1be84cac28775e9a7eec44e9a6f7495a412da66dff6 Jul 08 12:02:10.028621 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.028594 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:10.033896 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:10.033153 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e8a84c_c0d3_4f7b_ab8d_2398adcf10a4.slice/crio-0173b12d1ec8965a35dc8ada3752a4743f018e9727363aba20c6cd3d6413821b WatchSource:0}: Error finding container 0173b12d1ec8965a35dc8ada3752a4743f018e9727363aba20c6cd3d6413821b: Status 404 returned error can't find the container with id 0173b12d1ec8965a35dc8ada3752a4743f018e9727363aba20c6cd3d6413821b Jul 08 12:02:10.035624 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.034366 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xgt62" Jul 08 12:02:10.039319 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:10.039279 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e45ff0_4e75_4b46_bed1_2d447ad8afc1.slice/crio-c643e704aaff7c9669a0ddf2792b287101792955c4ff7790274c3ce85a6447bb WatchSource:0}: Error finding container c643e704aaff7c9669a0ddf2792b287101792955c4ff7790274c3ce85a6447bb: Status 404 returned error can't find the container with id c643e704aaff7c9669a0ddf2792b287101792955c4ff7790274c3ce85a6447bb Jul 08 12:02:10.040801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.040766 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" Jul 08 12:02:10.048024 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.047998 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" Jul 08 12:02:10.052492 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.052455 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfjj8" Jul 08 12:02:10.066944 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:10.066904 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed760d9_54eb_4b77_aef5_a9133d175fd3.slice/crio-1f04a3d37b45f8f706166cd1dab9b5bc558e5157ade63af23c12d8a736004c41 WatchSource:0}: Error finding container 1f04a3d37b45f8f706166cd1dab9b5bc558e5157ade63af23c12d8a736004c41: Status 404 returned error can't find the container with id 1f04a3d37b45f8f706166cd1dab9b5bc558e5157ade63af23c12d8a736004c41 Jul 08 12:02:10.072885 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:10.072847 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adcca2c_870d_4702_907c_a9023968bdbc.slice/crio-a822319ec9e85c07758813a6b92face6848bbd3417bfe60ef395f8596b3e6cc7 WatchSource:0}: Error finding container a822319ec9e85c07758813a6b92face6848bbd3417bfe60ef395f8596b3e6cc7: Status 404 returned error can't find the container with id a822319ec9e85c07758813a6b92face6848bbd3417bfe60ef395f8596b3e6cc7 Jul 08 12:02:10.094954 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.094917 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 12:02:10.311699 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.311589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:10.311699 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.311642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:10.311978 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.311776 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:10.311978 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.311782 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:10.311978 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.311845 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:11.311825411 +0000 UTC m=+3.994878191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:10.311978 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.311865 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:11.31185527 +0000 UTC m=+3.994908054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:10.513350 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.513313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:10.513571 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.513541 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:10.513571 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.513577 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:10.513776 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.513589 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:10.513776 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.513653 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:11.513631691 +0000 UTC m=+4.196684480 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:10.743341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.743213 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 11:57:09 +0000 UTC" deadline="2028-01-28 13:36:48.681990262 +0000 UTC" Jul 08 12:02:10.743341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.743287 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13657h34m37.938708436s" Jul 08 12:02:10.881034 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.880999 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:10.881256 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:10.881156 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:10.896541 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.896502 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xgt62" event={"ID":"453e570f-3d3f-4bdf-8475-dcf1b0bda30e","Type":"ContainerStarted","Data":"acf891bf498722bf0ae049ef19462d9a1f191495a04bd7b2ceefd0d84180a190"} Jul 08 12:02:10.898610 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.898557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ndns4" event={"ID":"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4","Type":"ContainerStarted","Data":"1c849211ef4b85b2daa3398150ff27234e06dae79aa590571948077d9aa4dcbd"} Jul 08 12:02:10.900659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.900621 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"c643e704aaff7c9669a0ddf2792b287101792955c4ff7790274c3ce85a6447bb"} Jul 08 12:02:10.902197 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.902128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dmgs9" event={"ID":"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c","Type":"ContainerStarted","Data":"ed243346d1be2ebb9581b71c1848ab53f572f30bb879fcb30c5eefef72cfa315"} Jul 08 12:02:10.903583 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.903550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfjj8" event={"ID":"5adcca2c-870d-4702-907c-a9023968bdbc","Type":"ContainerStarted","Data":"a822319ec9e85c07758813a6b92face6848bbd3417bfe60ef395f8596b3e6cc7"} Jul 08 12:02:10.906575 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.906537 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" event={"ID":"9ed760d9-54eb-4b77-aef5-a9133d175fd3","Type":"ContainerStarted","Data":"1f04a3d37b45f8f706166cd1dab9b5bc558e5157ade63af23c12d8a736004c41"} Jul 08 12:02:10.908273 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.908237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" event={"ID":"62dba4ee-bfa1-4812-8000-17d4e4377bd2","Type":"ContainerStarted","Data":"245de9697628a9687a8c2c304068ea296f6b71fe04af7df8c9a43c8bb2e32e22"} Jul 08 12:02:10.909935 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.909881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c4brj" event={"ID":"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4","Type":"ContainerStarted","Data":"0173b12d1ec8965a35dc8ada3752a4743f018e9727363aba20c6cd3d6413821b"} Jul 08 12:02:10.911229 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:10.911203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerStarted","Data":"f161022e3105d0871e74a1be84cac28775e9a7eec44e9a6f7495a412da66dff6"} Jul 08 12:02:11.321871 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.321829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:11.322078 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.321910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:11.322078 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.322000 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:11.322187 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.322074 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:11.322187 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.322090 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:13.322068605 +0000 UTC m=+6.005121387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:11.322187 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.322141 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:13.322121542 +0000 UTC m=+6.005174320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:11.523590 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.523549 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:11.523848 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.523829 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:11.523960 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.523855 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:11.523960 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.523868 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:11.523960 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.523951 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:13.523929208 +0000 UTC m=+6.206982006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:11.744449 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.744294 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 11:57:09 +0000 UTC" deadline="2027-12-11 10:02:50.367916199 +0000 UTC" Jul 08 12:02:11.744449 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.744375 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12502h0m38.62354791s" Jul 08 12:02:11.883740 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.882808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:11.883740 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.882934 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:11.883740 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:11.883321 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:11.883740 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:11.883390 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:12.880792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:12.880757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:12.881281 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:12.880871 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:13.341304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:13.340425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:13.341304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:13.340486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:13.341304 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.340631 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:13.341304 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.340698 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:17.340678227 +0000 UTC m=+10.023731007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:13.341304 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.341130 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:13.341304 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.341186 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:17.341168601 +0000 UTC m=+10.024221379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:13.542343 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:13.542301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:13.542517 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.542481 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:13.542517 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.542509 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:13.542595 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.542523 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:13.542595 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.542585 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:17.542569761 +0000 UTC m=+10.225622537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:13.881421 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:13.880908 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:13.881421 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.881054 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:13.881421 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:13.881113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:13.881421 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:13.881238 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:14.881745 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:14.881012 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:14.881745 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:14.881259 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:15.881268 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:15.880987 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:15.881439 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:15.881036 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:15.881439 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:15.881301 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:15.881439 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:15.881389 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:16.880605 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:16.880568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:16.881113 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:16.880686 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:17.378324 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:17.378284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:17.378538 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:17.378384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:17.378538 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.378490 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:17.378538 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.378510 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:17.378693 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.378574 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:25.378550525 +0000 UTC m=+18.061603304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:17.378693 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.378599 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:25.378586774 +0000 UTC m=+18.061639554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:17.579721 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:17.579626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:17.579909 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.579787 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:17.579909 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.579805 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:17.579909 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.579817 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:17.579909 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.579871 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:25.579852591 +0000 UTC m=+18.262905385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:17.882153 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:17.881818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:17.882153 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.881972 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:17.882153 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:17.881986 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:17.882153 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:17.882072 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:18.881309 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:18.881274 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:18.881508 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:18.881419 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:19.881043 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:19.881004 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:19.881488 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:19.881125 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:19.881680 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:19.881648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:19.881820 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:19.881742 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:20.880741 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:20.880706 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:20.880934 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:20.880860 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:21.880623 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:21.880584 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:21.881134 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:21.880600 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:21.881134 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:21.880770 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:21.881134 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:21.880918 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:22.881345 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:22.881301 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:22.881764 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:22.881446 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:23.881456 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:23.881416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:23.881945 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:23.881553 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:23.881945 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:23.881933 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:23.882057 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:23.882034 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:24.880668 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:24.880629 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:24.880874 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:24.880769 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:25.445484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:25.445435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:25.445979 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:25.445504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:25.445979 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.445617 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:25.445979 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.445685 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:41.445667678 +0000 UTC m=+34.128720453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:25.445979 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.445617 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:25.445979 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.445749 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:41.445735507 +0000 UTC m=+34.128788285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:25.647393 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:25.647349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:25.647562 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.647526 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:25.647562 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.647547 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:25.647562 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.647556 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:25.647665 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.647610 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:41.647595396 +0000 UTC m=+34.330648172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:25.880711 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:25.880607 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:25.880877 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:25.880607 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:25.880877 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.880758 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:25.880877 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:25.880814 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:26.880655 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:26.880605 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:26.881113 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:26.880753 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:27.884316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:27.884285 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:27.884703 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:27.884380 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:27.884703 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:27.884680 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:27.884772 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:27.884737 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:28.880723 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:28.880684 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:28.880969 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:28.880815 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:29.891834 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:29.891806 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:29.892231 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:29.891974 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:29.892231 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:29.892031 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:29.892231 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:29.892140 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:30.880959 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.880703 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:30.881296 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:30.881020 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:30.966018 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.965982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" event={"ID":"62dba4ee-bfa1-4812-8000-17d4e4377bd2","Type":"ContainerStarted","Data":"7a181c9aac493ca22b6ccadfd6ad3ed2ef9416328ae202cd4164e20ca719633e"} Jul 08 12:02:30.967663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.967633 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c4brj" event={"ID":"d6e8a84c-c0d3-4f7b-ab8d-2398adcf10a4","Type":"ContainerStarted","Data":"0973f4c705ea3eb2cabd576a2e3f26d56753b78d7b8876f9f941acc581711af6"} Jul 08 12:02:30.971107 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971084 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:02:30.971450 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971430 2560 generic.go:356] "Generic (PLEG): container finished" podID="03e45ff0-4e75-4b46-bed1-2d447ad8afc1" containerID="f8aa96a12540afb0f4f60f67e577211767e2fa4302e05998f0fd20cba8a49cea" exitCode=1 Jul 08 12:02:30.971532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"b58b529dcbe86e516fb19414bdc17e17531cfb2077d06b0fda9ef412e6626475"} Jul 08 12:02:30.971659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"afeaba6b2efb51d9d8a168455aa3aefc2dfd84e1cdfba78a3ae59ead0e4e453d"} Jul 08 12:02:30.971659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971638 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"f3f79a52fad361d736d41aa2732ec63228b5eebf6941f16e55911613b2c9eecb"} Jul 08 12:02:30.971775 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"87e8adbdd663468352fa8d2315487880b509925a8bc723f5178c5824bfedf3fc"} Jul 08 12:02:30.971775 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerDied","Data":"f8aa96a12540afb0f4f60f67e577211767e2fa4302e05998f0fd20cba8a49cea"} Jul 08 12:02:30.971924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.971780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"9437e46d1888bda252a27da1b3c23ef22ff4f5dad3814aa5fa3100a219829d08"} Jul 08 12:02:30.973660 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.973635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" event={"ID":"bbb64130c7b615dfb98b4e6af62c0947","Type":"ContainerStarted","Data":"b97c98089399d72ee6c34e5f63cb5733c79635722be46c43056c6af2bfaa0670"} Jul 08 12:02:30.987622 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:30.987567 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9tdgl" podStartSLOduration=3.269726383 podStartE2EDuration="22.987551658s" totalImagesPullingTime="19.717825275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:30.987315542 +0000 UTC m=+23.670368340" watchObservedRunningTime="2026-07-08 12:02:30.987551658 +0000 UTC m=+23.670604456" Jul 08 12:02:31.003395 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.003345 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" podStartSLOduration=22.00332901 podStartE2EDuration="22.00332901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:31.003168457 +0000 UTC m=+23.686221254" watchObservedRunningTime="2026-07-08 12:02:31.00332901 +0000 UTC m=+23.686381806" Jul 08 12:02:31.028008 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.027958 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c4brj" podStartSLOduration=3.211039546 podStartE2EDuration="23.027941933s" totalImagesPullingTime="19.816902387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:31.027483307 +0000 UTC m=+23.710536108" watchObservedRunningTime="2026-07-08 12:02:31.027941933 +0000 UTC m=+23.710994730" Jul 08 12:02:31.881453 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.881243 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:31.881625 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.881299 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:31.881625 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:31.881535 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:31.881625 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:31.881607 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:31.977546 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.977501 2560 generic.go:356] "Generic (PLEG): container finished" podID="7369164f-0ab7-4a79-ab87-1d7a5c77ef21" containerID="898ac8fd00992f1a43caeaaf51982232551c0af2582ea6a9e16873d6140dfdcf" exitCode=0 Jul 08 12:02:31.978059 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.977587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerDied","Data":"898ac8fd00992f1a43caeaaf51982232551c0af2582ea6a9e16873d6140dfdcf"} Jul 08 12:02:31.979341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.979312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xgt62" event={"ID":"453e570f-3d3f-4bdf-8475-dcf1b0bda30e","Type":"ContainerStarted","Data":"ca05c5a9d6e49e10c2d6009c98bf5db3308c94efed8a4941ce375309d1388cbd"} Jul 08 12:02:31.980819 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.980793 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ndns4" event={"ID":"eb3c4f96-1c10-47f9-b03c-76bebeef9fa4","Type":"ContainerStarted","Data":"7cc4210c374962da1c331b15ecb601dc4494c6e2aa1a8a221e8c8e88f808084b"} Jul 08 12:02:31.982261 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.982237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dmgs9" event={"ID":"0f1897f6-14e3-4e50-b3a6-6c15c91ae66c","Type":"ContainerStarted","Data":"d588bc7bb594ebecc882bef25d86885550c412b0d277644c23a2b3deb523c214"} Jul 08 12:02:31.983653 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.983627 2560 generic.go:356] "Generic (PLEG): container finished" podID="d6d9a6fed735c32fc3e333fe15459579" containerID="0e5a385395e15f2756e6452c07f1df9b0a210ad9d02171c19eb18ec3f5b5a25c" exitCode=0 Jul 08 12:02:31.983766 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.983700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" event={"ID":"d6d9a6fed735c32fc3e333fe15459579","Type":"ContainerDied","Data":"0e5a385395e15f2756e6452c07f1df9b0a210ad9d02171c19eb18ec3f5b5a25c"} Jul 08 12:02:31.985155 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.985128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfjj8" event={"ID":"5adcca2c-870d-4702-907c-a9023968bdbc","Type":"ContainerStarted","Data":"b09665c4617d3cd38e9bb41458c01339688a1dde704a30dbe873adc77c6e48cf"} Jul 08 12:02:31.986728 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:31.986704 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" event={"ID":"9ed760d9-54eb-4b77-aef5-a9133d175fd3","Type":"ContainerStarted","Data":"ba3c357f4c4a815b99fbb4f4ca18fd9deffa843c702aa88fe507c4df6e252f08"} Jul 08 12:02:32.019848 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.019794 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dmgs9" podStartSLOduration=15.322793517 podStartE2EDuration="24.01977478s" totalImagesPullingTime="8.696981263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:32.019759117 +0000 UTC m=+24.702811915" watchObservedRunningTime="2026-07-08 12:02:32.01977478 +0000 UTC m=+24.702827578" Jul 08 12:02:32.036201 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.036152 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xgt62" podStartSLOduration=4.312491399 podStartE2EDuration="24.036136238s" totalImagesPullingTime="19.723644839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:32.035450978 +0000 UTC m=+24.718503776" watchObservedRunningTime="2026-07-08 12:02:32.036136238 +0000 UTC m=+24.719189036" Jul 08 12:02:32.070558 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.070486 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ndns4" podStartSLOduration=4.367535827 podStartE2EDuration="24.070463574s" totalImagesPullingTime="19.702927747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:32.069688701 +0000 UTC m=+24.752741498" watchObservedRunningTime="2026-07-08 12:02:32.070463574 +0000 UTC m=+24.753516371" Jul 08 12:02:32.085996 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.085933 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tfjj8" podStartSLOduration=4.379508595 podStartE2EDuration="24.085912687s" totalImagesPullingTime="19.706404092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:32.085455878 +0000 UTC m=+24.768508677" watchObservedRunningTime="2026-07-08 12:02:32.085912687 +0000 UTC m=+24.768965487" Jul 08 12:02:32.379215 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.379187 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 12:02:32.776004 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.775879 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T12:02:32.379207653Z","UUID":"4b9a52b8-781a-44f8-ace6-e414cdd2f137","Handler":null,"Name":"","Endpoint":""} Jul 08 12:02:32.777482 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.777461 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 12:02:32.777620 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.777496 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 12:02:32.881208 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.881177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:32.881339 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:32.881319 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:32.991995 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.991953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" event={"ID":"d6d9a6fed735c32fc3e333fe15459579","Type":"ContainerStarted","Data":"3ff7699000389f87b13868a6befb8c8d96bb3019e80305084d843b2014c8fa55"} Jul 08 12:02:32.994754 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.994719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" event={"ID":"9ed760d9-54eb-4b77-aef5-a9133d175fd3","Type":"ContainerStarted","Data":"b5953adfc60f30a3bd6cefa837b982881558829db04b76fe0799d25e36757955"} Jul 08 12:02:32.998560 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.998535 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:02:32.999311 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:32.999274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"b8fccb0dd2334490bdc15cc57c4b93a3e2674436f5ff4f6c30a24b55781ac21d"} Jul 08 12:02:33.010481 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:33.010416 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" podStartSLOduration=24.010400223 podStartE2EDuration="24.010400223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:33.009717236 +0000 UTC m=+25.692770035" watchObservedRunningTime="2026-07-08 12:02:33.010400223 +0000 UTC m=+25.693453020" Jul 08 12:02:33.881255 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:33.881229 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:33.881446 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:33.881345 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:33.881446 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:33.881394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:33.881558 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:33.881490 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:34.004471 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:34.004430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" event={"ID":"9ed760d9-54eb-4b77-aef5-a9133d175fd3","Type":"ContainerStarted","Data":"17bffd81f05cb32e9c634c03d13e90ed152bf47c7770c87a74d029a5d779c87b"} Jul 08 12:02:34.023842 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:34.023781 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bc45v" podStartSLOduration=12.897391839 podStartE2EDuration="26.023762654s" totalImagesPullingTime="13.126370815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:34.023115364 +0000 UTC m=+26.706168164" watchObservedRunningTime="2026-07-08 12:02:34.023762654 +0000 UTC m=+26.706815463" Jul 08 12:02:34.881460 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:34.881278 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:34.881658 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:34.881562 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:35.538055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:35.538012 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:35.539015 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:35.538825 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:35.880963 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:35.880832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:35.880963 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:35.880880 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:35.881182 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:35.880987 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:35.881182 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:35.881100 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:36.011840 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:36.011809 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:02:36.012290 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:36.012266 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"09af0ac8c66a58a3c71160d148e9af27b7748077774b7b1b439efc40fe85366c"} Jul 08 12:02:36.012562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:36.012541 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:36.012644 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:36.012631 2560 scope.go:117] "RemoveContainer" containerID="f8aa96a12540afb0f4f60f67e577211767e2fa4302e05998f0fd20cba8a49cea" Jul 08 12:02:36.013022 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:36.013004 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ndns4" Jul 08 12:02:36.881354 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:36.881322 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:36.881828 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:36.881454 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:37.020314 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.020214 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:02:37.020807 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.020772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" event={"ID":"03e45ff0-4e75-4b46-bed1-2d447ad8afc1","Type":"ContainerStarted","Data":"01c4efba0e98fc841d76ec26b2032098846af4055c9d64caa96f6fcd967236ef"} Jul 08 12:02:37.021703 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.021679 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:37.021808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.021757 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:37.021808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.021803 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:37.043096 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.043064 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:37.043774 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.043749 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:02:37.053185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.053127 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" podStartSLOduration=9.258954429 podStartE2EDuration="29.053108181s" totalImagesPullingTime="19.794153752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:37.052505298 +0000 UTC m=+29.735558096" watchObservedRunningTime="2026-07-08 12:02:37.053108181 +0000 UTC m=+29.736160980" Jul 08 12:02:37.220791 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.220693 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rc5xg"] Jul 08 12:02:37.220987 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.220903 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:37.221046 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:37.221009 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:37.225009 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.224972 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rnpfb"] Jul 08 12:02:37.225197 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.225046 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f8lqw"] Jul 08 12:02:37.225197 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.225148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:37.225300 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:37.225258 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:37.225357 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:37.225335 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:37.225436 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:37.225413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:38.880761 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:38.880721 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:38.881183 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:38.880823 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:38.881183 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:38.880829 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:38.881183 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:38.880852 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:38.881183 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:38.880943 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:38.881183 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:38.881022 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:39.029462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:39.029431 2560 generic.go:356] "Generic (PLEG): container finished" podID="7369164f-0ab7-4a79-ab87-1d7a5c77ef21" containerID="deef6f6a9a3d1093d67966f35e1d8b36cdb3948a9ae82757e4a8bcbe5bf8823b" exitCode=0 Jul 08 12:02:39.029593 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:39.029510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerDied","Data":"deef6f6a9a3d1093d67966f35e1d8b36cdb3948a9ae82757e4a8bcbe5bf8823b"} Jul 08 12:02:40.034843 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:40.034635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerStarted","Data":"a762cecae7d4888e7787a590036c761646f1fda82678fbef9942865389e561ac"} Jul 08 12:02:40.880714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:40.880671 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:40.880714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:40.880691 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:40.881007 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:40.880684 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:40.881007 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:40.880781 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:40.881007 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:40.880849 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:40.881007 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:40.880968 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:41.038532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:41.038491 2560 generic.go:356] "Generic (PLEG): container finished" podID="7369164f-0ab7-4a79-ab87-1d7a5c77ef21" containerID="a762cecae7d4888e7787a590036c761646f1fda82678fbef9942865389e561ac" exitCode=0 Jul 08 12:02:41.039034 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:41.038584 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerDied","Data":"a762cecae7d4888e7787a590036c761646f1fda82678fbef9942865389e561ac"} Jul 08 12:02:41.473124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:41.473082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:41.473124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:41.473124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:41.473446 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.473235 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:41.473446 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.473236 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:41.473446 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.473293 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:13.473278085 +0000 UTC m=+66.156330861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 12:02:41.473446 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.473307 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret podName:ced800b5-7cf8-42d6-b6dc-c8cd5898c813 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:13.47330123 +0000 UTC m=+66.156354006 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret") pod "global-pull-secret-syncer-rc5xg" (UID: "ced800b5-7cf8-42d6-b6dc-c8cd5898c813") : object "kube-system"/"original-pull-secret" not registered Jul 08 12:02:41.675001 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:41.674908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:41.675164 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.675118 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 12:02:41.675164 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.675146 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 12:02:41.675164 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.675159 2560 projected.go:196] Error preparing data for projected volume kube-api-access-fzt5d for pod openshift-network-diagnostics/network-check-target-rnpfb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:41.675283 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:41.675223 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d podName:15a64e87-733c-44f5-987c-5e544d1fcac1 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:13.675208347 +0000 UTC m=+66.358261122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzt5d" (UniqueName: "kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d") pod "network-check-target-rnpfb" (UID: "15a64e87-733c-44f5-987c-5e544d1fcac1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 12:02:42.881384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:42.881344 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:42.881937 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:42.881391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:42.881937 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:42.881343 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:42.881937 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:42.881467 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rc5xg" podUID="ced800b5-7cf8-42d6-b6dc-c8cd5898c813" Jul 08 12:02:42.881937 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:42.881525 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8lqw" podUID="cd5e2c61-626f-4c5b-bcba-91ec3c1073d4" Jul 08 12:02:42.881937 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:42.881602 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnpfb" podUID="15a64e87-733c-44f5-987c-5e544d1fcac1" Jul 08 12:02:43.045310 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.045229 2560 generic.go:356] "Generic (PLEG): container finished" podID="7369164f-0ab7-4a79-ab87-1d7a5c77ef21" containerID="e0ceb80373788e52f101dc807243166ce72ad71203492ec676b0b43e4fd4cd30" exitCode=0 Jul 08 12:02:43.045310 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.045280 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerDied","Data":"e0ceb80373788e52f101dc807243166ce72ad71203492ec676b0b43e4fd4cd30"} Jul 08 12:02:43.048412 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.048392 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeReady" Jul 08 12:02:43.048532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.048521 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 12:02:43.098356 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.098322 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4x886"] Jul 08 12:02:43.100905 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.100855 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.101100 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.101080 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wg898"] Jul 08 12:02:43.103579 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.103558 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.103710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.103639 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 12:02:43.103710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.103651 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 12:02:43.103825 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.103653 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5pzr4\"" Jul 08 12:02:43.107317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.107204 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-twvk9\"" Jul 08 12:02:43.107317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.107213 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 12:02:43.107317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.107260 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 12:02:43.107317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.107271 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 12:02:43.111453 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.111385 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4x886"] Jul 08 12:02:43.114104 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.114075 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wg898"] Jul 08 12:02:43.186217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.186176 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lkpc\" (UniqueName: \"kubernetes.io/projected/d593b87b-d888-4119-a222-83aebe7c1b73-kube-api-access-9lkpc\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.186413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.186229 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d593b87b-d888-4119-a222-83aebe7c1b73-config-volume\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.186413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.186248 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.186413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.186282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d593b87b-d888-4119-a222-83aebe7c1b73-tmp-dir\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.287508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.287457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txj47\" (UniqueName: \"kubernetes.io/projected/e24f66af-36b0-4c3e-8baf-c6306b40575b-kube-api-access-txj47\") pod \"ingress-canary-wg898\" (UID: \"e24f66af-36b0-4c3e-8baf-c6306b40575b\") " pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.287687 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.287539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lkpc\" (UniqueName: \"kubernetes.io/projected/d593b87b-d888-4119-a222-83aebe7c1b73-kube-api-access-9lkpc\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.287687 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.287559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e24f66af-36b0-4c3e-8baf-c6306b40575b-cert\") pod \"ingress-canary-wg898\" (UID: \"e24f66af-36b0-4c3e-8baf-c6306b40575b\") " pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.287687 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.287575 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d593b87b-d888-4119-a222-83aebe7c1b73-config-volume\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.287687 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.287592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.287687 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.287648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d593b87b-d888-4119-a222-83aebe7c1b73-tmp-dir\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.287687 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:43.287680 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:02:43.287950 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:43.287732 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:43.787712678 +0000 UTC m=+36.470765453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:02:43.288067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.288044 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d593b87b-d888-4119-a222-83aebe7c1b73-tmp-dir\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.288173 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.288155 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d593b87b-d888-4119-a222-83aebe7c1b73-config-volume\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.299189 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.299112 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lkpc\" (UniqueName: \"kubernetes.io/projected/d593b87b-d888-4119-a222-83aebe7c1b73-kube-api-access-9lkpc\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.388907 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.388841 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txj47\" (UniqueName: \"kubernetes.io/projected/e24f66af-36b0-4c3e-8baf-c6306b40575b-kube-api-access-txj47\") pod \"ingress-canary-wg898\" (UID: \"e24f66af-36b0-4c3e-8baf-c6306b40575b\") " pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.389088 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.388940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e24f66af-36b0-4c3e-8baf-c6306b40575b-cert\") pod \"ingress-canary-wg898\" (UID: \"e24f66af-36b0-4c3e-8baf-c6306b40575b\") " pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.391547 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.391510 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e24f66af-36b0-4c3e-8baf-c6306b40575b-cert\") pod \"ingress-canary-wg898\" (UID: \"e24f66af-36b0-4c3e-8baf-c6306b40575b\") " pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.398050 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.398022 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txj47\" (UniqueName: \"kubernetes.io/projected/e24f66af-36b0-4c3e-8baf-c6306b40575b-kube-api-access-txj47\") pod \"ingress-canary-wg898\" (UID: \"e24f66af-36b0-4c3e-8baf-c6306b40575b\") " pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.420142 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.420099 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wg898" Jul 08 12:02:43.565233 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.565128 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wg898"] Jul 08 12:02:43.568954 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:02:43.568925 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24f66af_36b0_4c3e_8baf_c6306b40575b.slice/crio-c608cb828308e40817aef926675414d40d3034bb57e2005e53c42ac08b17da11 WatchSource:0}: Error finding container c608cb828308e40817aef926675414d40d3034bb57e2005e53c42ac08b17da11: Status 404 returned error can't find the container with id c608cb828308e40817aef926675414d40d3034bb57e2005e53c42ac08b17da11 Jul 08 12:02:43.792745 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:43.792704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:43.792970 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:43.792861 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:02:43.792970 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:43.792949 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:44.79293263 +0000 UTC m=+37.475985406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:02:44.049005 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.048916 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wg898" event={"ID":"e24f66af-36b0-4c3e-8baf-c6306b40575b","Type":"ContainerStarted","Data":"c608cb828308e40817aef926675414d40d3034bb57e2005e53c42ac08b17da11"} Jul 08 12:02:44.798824 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.798768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:44.799138 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:44.798948 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:02:44.799138 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:44.799026 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:46.799002568 +0000 UTC m=+39.482055347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:02:44.881269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.881217 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:02:44.881456 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.881230 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:02:44.881456 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.881350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:02:44.884140 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.884109 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 12:02:44.885414 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.885391 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 12:02:44.885536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.885443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvrx5\"" Jul 08 12:02:44.885536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.885477 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 12:02:44.885714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.885700 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 12:02:44.885774 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:44.885723 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rdkjt\"" Jul 08 12:02:46.816536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:46.816504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:46.817123 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:46.816644 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:02:46.817123 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:46.816716 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:50.816697293 +0000 UTC m=+43.499750069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:02:48.061284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:48.061245 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wg898" event={"ID":"e24f66af-36b0-4c3e-8baf-c6306b40575b","Type":"ContainerStarted","Data":"4139d439bc1c473195b696409d3f379f501d5f8b97e6b89d645b2c05f647e5f6"} Jul 08 12:02:48.078673 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:48.078604 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wg898" podStartSLOduration=1.357135843 podStartE2EDuration="5.078581338s" totalImagesPullingTime="3.721445495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:48.078311877 +0000 UTC m=+40.761364695" watchObservedRunningTime="2026-07-08 12:02:48.078581338 +0000 UTC m=+40.761634139" Jul 08 12:02:50.849992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:50.849944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:50.850410 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:50.850072 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:02:50.850410 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:50.850129 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:02:58.850114871 +0000 UTC m=+51.533167646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:02:52.073906 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:52.073862 2560 generic.go:356] "Generic (PLEG): container finished" podID="7369164f-0ab7-4a79-ab87-1d7a5c77ef21" containerID="16e5085b80162a0c693161a66c825f1f141f58eb6f42f78953a1c7c2263d7d7c" exitCode=0 Jul 08 12:02:52.074271 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:52.073927 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerDied","Data":"16e5085b80162a0c693161a66c825f1f141f58eb6f42f78953a1c7c2263d7d7c"} Jul 08 12:02:53.079970 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:53.079932 2560 generic.go:356] "Generic (PLEG): container finished" podID="7369164f-0ab7-4a79-ab87-1d7a5c77ef21" containerID="90e918f061a6037504cf3edae0632ce0f31d5bf69f653c9970f2e9d480edbb7a" exitCode=0 Jul 08 12:02:53.080353 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:53.080002 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerDied","Data":"90e918f061a6037504cf3edae0632ce0f31d5bf69f653c9970f2e9d480edbb7a"} Jul 08 12:02:54.086336 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:54.086297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" event={"ID":"7369164f-0ab7-4a79-ab87-1d7a5c77ef21","Type":"ContainerStarted","Data":"854c4a2eee7836f9d929ee95409b3727c7df6a663c2b5333bfe6d647e33642c8"} Jul 08 12:02:54.111634 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:54.111571 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tmvlb" podStartSLOduration=19.098673255 podStartE2EDuration="46.111549793s" totalImagesPullingTime="27.012876538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:02:54.110368004 +0000 UTC m=+46.793420802" watchObservedRunningTime="2026-07-08 12:02:54.111549793 +0000 UTC m=+46.794602596" Jul 08 12:02:58.908908 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:02:58.908843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:02:58.909367 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:58.909011 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:02:58.909367 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:02:58.909080 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:14.909063641 +0000 UTC m=+67.592116417 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:03:09.041507 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:09.041478 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkcwp" Jul 08 12:03:13.516370 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.516328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:03:13.516370 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.516371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:03:13.519253 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.519232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 12:03:13.519321 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.519266 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 12:03:13.527244 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:13.527219 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 12:03:13.527330 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:13.527320 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs podName:cd5e2c61-626f-4c5b-bcba-91ec3c1073d4 nodeName:}" failed. No retries permitted until 2026-07-08 12:04:17.527301154 +0000 UTC m=+130.210353930 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs") pod "network-metrics-daemon-f8lqw" (UID: "cd5e2c61-626f-4c5b-bcba-91ec3c1073d4") : secret "metrics-daemon-secret" not found Jul 08 12:03:13.531181 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.531158 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced800b5-7cf8-42d6-b6dc-c8cd5898c813-original-pull-secret\") pod \"global-pull-secret-syncer-rc5xg\" (UID: \"ced800b5-7cf8-42d6-b6dc-c8cd5898c813\") " pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:03:13.707329 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.707295 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rc5xg" Jul 08 12:03:13.718721 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.718669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:03:13.721613 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.721582 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 12:03:13.731413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.731382 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 12:03:13.742043 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.742014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzt5d\" (UniqueName: \"kubernetes.io/projected/15a64e87-733c-44f5-987c-5e544d1fcac1-kube-api-access-fzt5d\") pod \"network-check-target-rnpfb\" (UID: \"15a64e87-733c-44f5-987c-5e544d1fcac1\") " pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:03:13.846650 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.846606 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rc5xg"] Jul 08 12:03:13.853626 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:13.853584 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced800b5_7cf8_42d6_b6dc_c8cd5898c813.slice/crio-228ce1d8d96dac99f621782a1af22806b44adb2b824467f998f76a8b21912683 WatchSource:0}: Error finding container 228ce1d8d96dac99f621782a1af22806b44adb2b824467f998f76a8b21912683: Status 404 returned error can't find the container with id 228ce1d8d96dac99f621782a1af22806b44adb2b824467f998f76a8b21912683 Jul 08 12:03:13.997218 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:13.997185 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvrx5\"" Jul 08 12:03:14.005464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:14.005433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:03:14.136863 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:14.136813 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rc5xg" event={"ID":"ced800b5-7cf8-42d6-b6dc-c8cd5898c813","Type":"ContainerStarted","Data":"228ce1d8d96dac99f621782a1af22806b44adb2b824467f998f76a8b21912683"} Jul 08 12:03:14.138163 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:14.138132 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rnpfb"] Jul 08 12:03:14.141259 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:14.141234 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a64e87_733c_44f5_987c_5e544d1fcac1.slice/crio-1d4ab0db293ce323ab266cd8b4ada1cbb2e94727a3aa014b715d8edccd050dec WatchSource:0}: Error finding container 1d4ab0db293ce323ab266cd8b4ada1cbb2e94727a3aa014b715d8edccd050dec: Status 404 returned error can't find the container with id 1d4ab0db293ce323ab266cd8b4ada1cbb2e94727a3aa014b715d8edccd050dec Jul 08 12:03:14.927774 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:14.927724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:03:14.928215 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:14.927853 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 12:03:14.928215 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:14.927933 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls podName:d593b87b-d888-4119-a222-83aebe7c1b73 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:46.927918382 +0000 UTC m=+99.610971158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls") pod "dns-default-4x886" (UID: "d593b87b-d888-4119-a222-83aebe7c1b73") : secret "dns-default-metrics-tls" not found Jul 08 12:03:15.141451 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:15.141400 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rnpfb" event={"ID":"15a64e87-733c-44f5-987c-5e544d1fcac1","Type":"ContainerStarted","Data":"1d4ab0db293ce323ab266cd8b4ada1cbb2e94727a3aa014b715d8edccd050dec"} Jul 08 12:03:20.157026 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:20.156975 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rc5xg" event={"ID":"ced800b5-7cf8-42d6-b6dc-c8cd5898c813","Type":"ContainerStarted","Data":"ddc9cdc4c7e1895e69d3e77316acef4c83987ae661999afa29a4f23e966f4084"} Jul 08 12:03:20.158576 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:20.158541 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rnpfb" event={"ID":"15a64e87-733c-44f5-987c-5e544d1fcac1","Type":"ContainerStarted","Data":"f5b95b0f5cf4c7977d55de6b5eebe4f206da42dfb17a5f16405e126f3429b5ec"} Jul 08 12:03:20.158710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:20.158611 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:03:20.175374 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:20.175319 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rc5xg" podStartSLOduration=66.285913827 podStartE2EDuration="1m12.175302352s" totalImagesPullingTime="5.889388525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:20.174830266 +0000 UTC m=+72.857883065" watchObservedRunningTime="2026-07-08 12:03:20.175302352 +0000 UTC m=+72.858355211" Jul 08 12:03:20.193587 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:20.193527 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rnpfb" podStartSLOduration=66.583224451 podStartE2EDuration="1m12.193513853s" totalImagesPullingTime="5.610289402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:20.192729406 +0000 UTC m=+72.875782204" watchObservedRunningTime="2026-07-08 12:03:20.193513853 +0000 UTC m=+72.876566630" Jul 08 12:03:23.486424 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.486389 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq"] Jul 08 12:03:23.488550 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.488530 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" Jul 08 12:03:23.491221 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.491193 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jhk9s\"" Jul 08 12:03:23.491371 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.491328 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 12:03:23.492286 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.492267 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 12:03:23.498833 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.498781 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq"] Jul 08 12:03:23.590165 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.590123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lwcc\" (UniqueName: \"kubernetes.io/projected/fd03dce9-1cb3-4a89-b65e-4828273956ee-kube-api-access-6lwcc\") pod \"volume-data-source-validator-5d456fdbcd-f2jpq\" (UID: \"fd03dce9-1cb3-4a89-b65e-4828273956ee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" Jul 08 12:03:23.604222 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.604184 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg"] Jul 08 12:03:23.606429 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.606405 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.622210 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.622175 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 12:03:23.622396 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.622285 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 12:03:23.622396 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.622288 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 12:03:23.623151 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.623127 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5w5lg\"" Jul 08 12:03:23.635776 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.635746 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 12:03:23.650211 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.650181 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg"] Jul 08 12:03:23.690909 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.690837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lwcc\" (UniqueName: \"kubernetes.io/projected/fd03dce9-1cb3-4a89-b65e-4828273956ee-kube-api-access-6lwcc\") pod \"volume-data-source-validator-5d456fdbcd-f2jpq\" (UID: \"fd03dce9-1cb3-4a89-b65e-4828273956ee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" Jul 08 12:03:23.690909 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.690907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85462481-e494-40d8-b1d0-959c40dce0b7-config\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.691124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.690935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qwt\" (UniqueName: \"kubernetes.io/projected/85462481-e494-40d8-b1d0-959c40dce0b7-kube-api-access-n7qwt\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.691124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.690964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85462481-e494-40d8-b1d0-959c40dce0b7-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.708145 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.707007 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d48bf998-dpvjz"] Jul 08 12:03:23.710725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.710696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.711273 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.711246 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lwcc\" (UniqueName: \"kubernetes.io/projected/fd03dce9-1cb3-4a89-b65e-4828273956ee-kube-api-access-6lwcc\") pod \"volume-data-source-validator-5d456fdbcd-f2jpq\" (UID: \"fd03dce9-1cb3-4a89-b65e-4828273956ee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" Jul 08 12:03:23.713086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.713064 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 12:03:23.713824 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.713799 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 12:03:23.713947 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.713799 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 12:03:23.713947 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.713939 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j8q6l\"" Jul 08 12:03:23.721094 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.721070 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 12:03:23.723850 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.723823 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d48bf998-dpvjz"] Jul 08 12:03:23.792262 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792232 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-trusted-ca\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792443 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-installation-pull-secrets\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792443 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48pk5\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-kube-api-access-48pk5\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792443 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792311 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-ca-trust-extracted\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792443 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-certificates\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-bound-sa-token\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792493 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85462481-e494-40d8-b1d0-959c40dce0b7-config\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.792564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qwt\" (UniqueName: \"kubernetes.io/projected/85462481-e494-40d8-b1d0-959c40dce0b7-kube-api-access-n7qwt\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.792564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85462481-e494-40d8-b1d0-959c40dce0b7-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.792679 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-image-registry-private-configuration\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.792679 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.792588 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.793107 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.793076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85462481-e494-40d8-b1d0-959c40dce0b7-config\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.794967 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.794945 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85462481-e494-40d8-b1d0-959c40dce0b7-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.797980 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.797955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" Jul 08 12:03:23.804752 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.804716 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qwt\" (UniqueName: \"kubernetes.io/projected/85462481-e494-40d8-b1d0-959c40dce0b7-kube-api-access-n7qwt\") pod \"service-ca-operator-5d46d9ffd4-glfxg\" (UID: \"85462481-e494-40d8-b1d0-959c40dce0b7\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.893785 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-image-registry-private-configuration\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.893785 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-trusted-ca\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893839 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-installation-pull-secrets\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48pk5\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-kube-api-access-48pk5\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-ca-trust-extracted\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:23.893978 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.893996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-certificates\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.894022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-bound-sa-token\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.894414 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:23.894001 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d48bf998-dpvjz: secret "image-registry-tls" not found Jul 08 12:03:23.894414 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:23.894345 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls podName:cfd28cfb-6eb0-491a-bf23-1b760b0cc25e nodeName:}" failed. No retries permitted until 2026-07-08 12:03:24.394321705 +0000 UTC m=+77.077374481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls") pod "image-registry-5d48bf998-dpvjz" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e") : secret "image-registry-tls" not found Jul 08 12:03:23.894989 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.894956 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-certificates\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.895315 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.895283 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-ca-trust-extracted\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.895938 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.895910 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-trusted-ca\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.896932 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.896881 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-image-registry-private-configuration\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.897064 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.897044 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-installation-pull-secrets\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.908530 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.908491 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-bound-sa-token\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.908982 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.908951 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48pk5\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-kube-api-access-48pk5\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:23.915602 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.915572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" Jul 08 12:03:23.934628 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:23.934590 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq"] Jul 08 12:03:23.939733 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:23.939693 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd03dce9_1cb3_4a89_b65e_4828273956ee.slice/crio-32e5c7a2a0b3de5f4fc3306bf20d80a805c8ea85e94337c8824f48b4f39050cc WatchSource:0}: Error finding container 32e5c7a2a0b3de5f4fc3306bf20d80a805c8ea85e94337c8824f48b4f39050cc: Status 404 returned error can't find the container with id 32e5c7a2a0b3de5f4fc3306bf20d80a805c8ea85e94337c8824f48b4f39050cc Jul 08 12:03:24.050579 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:24.050496 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg"] Jul 08 12:03:24.054094 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:24.054061 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85462481_e494_40d8_b1d0_959c40dce0b7.slice/crio-fd28eed63c6751d0ad37407674eae593dad4951032434e3861eab5a49fcbf5b4 WatchSource:0}: Error finding container fd28eed63c6751d0ad37407674eae593dad4951032434e3861eab5a49fcbf5b4: Status 404 returned error can't find the container with id fd28eed63c6751d0ad37407674eae593dad4951032434e3861eab5a49fcbf5b4 Jul 08 12:03:24.169346 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:24.169310 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" event={"ID":"fd03dce9-1cb3-4a89-b65e-4828273956ee","Type":"ContainerStarted","Data":"32e5c7a2a0b3de5f4fc3306bf20d80a805c8ea85e94337c8824f48b4f39050cc"} Jul 08 12:03:24.170372 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:24.170337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" event={"ID":"85462481-e494-40d8-b1d0-959c40dce0b7","Type":"ContainerStarted","Data":"fd28eed63c6751d0ad37407674eae593dad4951032434e3861eab5a49fcbf5b4"} Jul 08 12:03:24.398528 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:24.398438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:24.398699 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:24.398552 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 12:03:24.398699 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:24.398564 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d48bf998-dpvjz: secret "image-registry-tls" not found Jul 08 12:03:24.398699 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:24.398617 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls podName:cfd28cfb-6eb0-491a-bf23-1b760b0cc25e nodeName:}" failed. No retries permitted until 2026-07-08 12:03:25.398600914 +0000 UTC m=+78.081653690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls") pod "image-registry-5d48bf998-dpvjz" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e") : secret "image-registry-tls" not found Jul 08 12:03:25.406333 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:25.406287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:25.406999 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:25.406444 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 12:03:25.406999 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:25.406460 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d48bf998-dpvjz: secret "image-registry-tls" not found Jul 08 12:03:25.406999 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:25.406535 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls podName:cfd28cfb-6eb0-491a-bf23-1b760b0cc25e nodeName:}" failed. No retries permitted until 2026-07-08 12:03:27.406514782 +0000 UTC m=+80.089567558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls") pod "image-registry-5d48bf998-dpvjz" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e") : secret "image-registry-tls" not found Jul 08 12:03:26.177410 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:26.177375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" event={"ID":"fd03dce9-1cb3-4a89-b65e-4828273956ee","Type":"ContainerStarted","Data":"277ab49fc1b73588f54adced882a5ad99a6b02983a53fe9ae99a9d544ee7bb6c"} Jul 08 12:03:26.195649 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:26.195589 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f2jpq" podStartSLOduration=1.9706605179999999 podStartE2EDuration="3.195569435s" totalImagesPullingTime="1.224908917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:26.195096848 +0000 UTC m=+78.878149645" watchObservedRunningTime="2026-07-08 12:03:26.195569435 +0000 UTC m=+78.878622233" Jul 08 12:03:27.181975 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:27.181932 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" event={"ID":"85462481-e494-40d8-b1d0-959c40dce0b7","Type":"ContainerStarted","Data":"bc801929bb793f4be7bde9529162e34143b663c682097531da186764165e3d90"} Jul 08 12:03:27.200466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:27.200408 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" podStartSLOduration=1.552007382 podStartE2EDuration="4.200390862s" totalImagesPullingTime="2.64838348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:27.199877267 +0000 UTC m=+79.882930066" watchObservedRunningTime="2026-07-08 12:03:27.200390862 +0000 UTC m=+79.883443661" Jul 08 12:03:27.420962 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:27.420917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:27.421161 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:27.421050 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 12:03:27.421161 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:27.421071 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d48bf998-dpvjz: secret "image-registry-tls" not found Jul 08 12:03:27.421161 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:27.421130 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls podName:cfd28cfb-6eb0-491a-bf23-1b760b0cc25e nodeName:}" failed. No retries permitted until 2026-07-08 12:03:31.421114576 +0000 UTC m=+84.104167351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls") pod "image-registry-5d48bf998-dpvjz" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e") : secret "image-registry-tls" not found Jul 08 12:03:28.062955 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.062819 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4sn6j"] Jul 08 12:03:28.065835 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.065815 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.069518 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.069490 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 12:03:28.069518 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.069512 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 12:03:28.070369 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.070352 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6nfnf\"" Jul 08 12:03:28.070369 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.070362 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 12:03:28.070529 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.070513 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 12:03:28.080731 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.080697 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4sn6j"] Jul 08 12:03:28.126783 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.126736 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.126783 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.126783 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb192faa-37da-4e22-90dc-613baf99d463-data-volume\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.127055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.126811 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtt4t\" (UniqueName: \"kubernetes.io/projected/eb192faa-37da-4e22-90dc-613baf99d463-kube-api-access-dtt4t\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.127055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.126870 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.127055 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.126959 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb192faa-37da-4e22-90dc-613baf99d463-crio-socket\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228215 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228215 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb192faa-37da-4e22-90dc-613baf99d463-data-volume\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtt4t\" (UniqueName: \"kubernetes.io/projected/eb192faa-37da-4e22-90dc-613baf99d463-kube-api-access-dtt4t\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb192faa-37da-4e22-90dc-613baf99d463-crio-socket\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.228321 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.228409 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228417 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb192faa-37da-4e22-90dc-613baf99d463-crio-socket\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.228462 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:28.728441165 +0000 UTC m=+81.411493955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : configmap "kube-rbac-proxy" not found Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.228496 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:28.728477489 +0000 UTC m=+81.411530266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : secret "insights-runtime-extractor-tls" not found Jul 08 12:03:28.228741 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.228592 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb192faa-37da-4e22-90dc-613baf99d463-data-volume\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.244552 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.244521 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtt4t\" (UniqueName: \"kubernetes.io/projected/eb192faa-37da-4e22-90dc-613baf99d463-kube-api-access-dtt4t\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.736246 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.733501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.736246 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:28.733594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:28.736246 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.733791 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 12:03:28.736246 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.733864 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:29.733844259 +0000 UTC m=+82.416897035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : secret "insights-runtime-extractor-tls" not found Jul 08 12:03:28.736246 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.733960 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 12:03:28.736246 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:28.734006 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:29.733985893 +0000 UTC m=+82.417038671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : configmap "kube-rbac-proxy" not found Jul 08 12:03:29.741134 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.741081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:29.741134 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.741141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:29.741630 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:29.741213 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 12:03:29.741630 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:29.741272 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 12:03:29.741630 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:29.741291 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:31.741275066 +0000 UTC m=+84.424327842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : configmap "kube-rbac-proxy" not found Jul 08 12:03:29.741630 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:29.741327 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:31.74131246 +0000 UTC m=+84.424365237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : secret "insights-runtime-extractor-tls" not found Jul 08 12:03:29.986333 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.986293 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-nsvnf"] Jul 08 12:03:29.989696 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.989667 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:29.993207 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.993108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 12:03:29.993347 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.993211 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 12:03:29.994272 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.994250 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 12:03:29.994272 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.994263 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r9k2n\"" Jul 08 12:03:29.994442 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.994256 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 12:03:29.999664 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:29.999625 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-nsvnf"] Jul 08 12:03:30.044298 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.044253 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fc2f056e-f503-4db5-be74-7a9d5a7c308c-signing-key\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.044449 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.044328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fc2f056e-f503-4db5-be74-7a9d5a7c308c-signing-cabundle\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.044449 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.044349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhkp6\" (UniqueName: \"kubernetes.io/projected/fc2f056e-f503-4db5-be74-7a9d5a7c308c-kube-api-access-vhkp6\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.144852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.144805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fc2f056e-f503-4db5-be74-7a9d5a7c308c-signing-key\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.145047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.144937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fc2f056e-f503-4db5-be74-7a9d5a7c308c-signing-cabundle\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.145047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.144962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhkp6\" (UniqueName: \"kubernetes.io/projected/fc2f056e-f503-4db5-be74-7a9d5a7c308c-kube-api-access-vhkp6\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.145655 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.145636 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fc2f056e-f503-4db5-be74-7a9d5a7c308c-signing-cabundle\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.147659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.147626 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fc2f056e-f503-4db5-be74-7a9d5a7c308c-signing-key\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.155405 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.155378 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhkp6\" (UniqueName: \"kubernetes.io/projected/fc2f056e-f503-4db5-be74-7a9d5a7c308c-kube-api-access-vhkp6\") pod \"service-ca-758f599c5-nsvnf\" (UID: \"fc2f056e-f503-4db5-be74-7a9d5a7c308c\") " pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.254954 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.254858 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xgt62_453e570f-3d3f-4bdf-8475-dcf1b0bda30e/dns-node-resolver/0.log" Jul 08 12:03:30.299191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.299149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-nsvnf" Jul 08 12:03:30.439030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.438999 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-nsvnf"] Jul 08 12:03:30.442628 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:30.442588 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2f056e_f503_4db5_be74_7a9d5a7c308c.slice/crio-6556fac5fc140993af58803b36dec624be087501d2a50aa9aba9c9b00b7704a6 WatchSource:0}: Error finding container 6556fac5fc140993af58803b36dec624be087501d2a50aa9aba9c9b00b7704a6: Status 404 returned error can't find the container with id 6556fac5fc140993af58803b36dec624be087501d2a50aa9aba9c9b00b7704a6 Jul 08 12:03:30.852148 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:30.852119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dmgs9_0f1897f6-14e3-4e50-b3a6-6c15c91ae66c/node-ca/0.log" Jul 08 12:03:31.195800 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:31.195697 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-nsvnf" event={"ID":"fc2f056e-f503-4db5-be74-7a9d5a7c308c","Type":"ContainerStarted","Data":"9cadb71b0da2134ff2dd3178b3286fb879bcd664609104176d1d643e06e0950d"} Jul 08 12:03:31.195800 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:31.195758 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-nsvnf" event={"ID":"fc2f056e-f503-4db5-be74-7a9d5a7c308c","Type":"ContainerStarted","Data":"6556fac5fc140993af58803b36dec624be087501d2a50aa9aba9c9b00b7704a6"} Jul 08 12:03:31.457147 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:31.457047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:31.457300 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.457203 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 12:03:31.457300 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.457227 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d48bf998-dpvjz: secret "image-registry-tls" not found Jul 08 12:03:31.457300 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.457286 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls podName:cfd28cfb-6eb0-491a-bf23-1b760b0cc25e nodeName:}" failed. No retries permitted until 2026-07-08 12:03:39.457270007 +0000 UTC m=+92.140322783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls") pod "image-registry-5d48bf998-dpvjz" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e") : secret "image-registry-tls" not found Jul 08 12:03:31.760269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:31.760232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:31.760471 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:31.760294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:31.760471 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.760365 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 12:03:31.760471 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.760402 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 12:03:31.760471 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.760429 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:35.760413889 +0000 UTC m=+88.443466665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : configmap "kube-rbac-proxy" not found Jul 08 12:03:31.760471 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:31.760455 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:35.76043906 +0000 UTC m=+88.443491835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : secret "insights-runtime-extractor-tls" not found Jul 08 12:03:32.051773 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:32.051677 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wg898_e24f66af-36b0-4c3e-8baf-c6306b40575b/serve-healthcheck-canary/0.log" Jul 08 12:03:35.797027 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:35.796980 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:35.797027 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:35.797032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:35.797462 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:35.797117 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 12:03:35.797462 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:35.797173 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 12:03:35.797462 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:35.797186 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:43.797170098 +0000 UTC m=+96.480222878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : configmap "kube-rbac-proxy" not found Jul 08 12:03:35.797462 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:03:35.797218 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls podName:eb192faa-37da-4e22-90dc-613baf99d463 nodeName:}" failed. No retries permitted until 2026-07-08 12:03:43.79720662 +0000 UTC m=+96.480259401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4sn6j" (UID: "eb192faa-37da-4e22-90dc-613baf99d463") : secret "insights-runtime-extractor-tls" not found Jul 08 12:03:39.526709 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:39.526656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:39.529424 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:39.529397 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"image-registry-5d48bf998-dpvjz\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:39.625412 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:39.625368 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:39.771291 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:39.771228 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-nsvnf" podStartSLOduration=10.771205927 podStartE2EDuration="10.771205927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:31.217305487 +0000 UTC m=+83.900358276" watchObservedRunningTime="2026-07-08 12:03:39.771205927 +0000 UTC m=+92.454258726" Jul 08 12:03:39.771779 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:39.771762 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d48bf998-dpvjz"] Jul 08 12:03:39.775538 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:39.775511 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd28cfb_6eb0_491a_bf23_1b760b0cc25e.slice/crio-6bb7ca8b733eff169136de24a629d4ff8e8aa2dd74e8150766559ec1432cadb7 WatchSource:0}: Error finding container 6bb7ca8b733eff169136de24a629d4ff8e8aa2dd74e8150766559ec1432cadb7: Status 404 returned error can't find the container with id 6bb7ca8b733eff169136de24a629d4ff8e8aa2dd74e8150766559ec1432cadb7 Jul 08 12:03:40.227914 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:40.227800 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" event={"ID":"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e","Type":"ContainerStarted","Data":"6d2a35c5705d52c0418714dd02ddc6bc38e76ce5e31bd19dc1c684f299aa6ae7"} Jul 08 12:03:40.227914 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:40.227865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" event={"ID":"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e","Type":"ContainerStarted","Data":"6bb7ca8b733eff169136de24a629d4ff8e8aa2dd74e8150766559ec1432cadb7"} Jul 08 12:03:40.228120 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:40.227949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:03:40.251388 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:40.251331 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" podStartSLOduration=17.251312273 podStartE2EDuration="17.251312273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:03:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:40.249954624 +0000 UTC m=+92.933007421" watchObservedRunningTime="2026-07-08 12:03:40.251312273 +0000 UTC m=+92.934365071" Jul 08 12:03:43.865541 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:43.865496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:43.866172 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:43.865579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:43.866172 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:43.866132 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb192faa-37da-4e22-90dc-613baf99d463-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:43.868241 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:43.868213 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb192faa-37da-4e22-90dc-613baf99d463-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4sn6j\" (UID: \"eb192faa-37da-4e22-90dc-613baf99d463\") " pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:43.974598 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:43.974551 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4sn6j" Jul 08 12:03:44.114570 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:44.114532 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4sn6j"] Jul 08 12:03:44.117864 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:44.117836 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb192faa_37da_4e22_90dc_613baf99d463.slice/crio-18926ce925f1e2d68ffc559f9e78f0c40150e11aa6b05fb8d527174a39d3845a WatchSource:0}: Error finding container 18926ce925f1e2d68ffc559f9e78f0c40150e11aa6b05fb8d527174a39d3845a: Status 404 returned error can't find the container with id 18926ce925f1e2d68ffc559f9e78f0c40150e11aa6b05fb8d527174a39d3845a Jul 08 12:03:44.241313 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:44.241271 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4sn6j" event={"ID":"eb192faa-37da-4e22-90dc-613baf99d463","Type":"ContainerStarted","Data":"18926ce925f1e2d68ffc559f9e78f0c40150e11aa6b05fb8d527174a39d3845a"} Jul 08 12:03:46.996673 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:46.996581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:03:46.999125 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:46.999096 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d593b87b-d888-4119-a222-83aebe7c1b73-metrics-tls\") pod \"dns-default-4x886\" (UID: \"d593b87b-d888-4119-a222-83aebe7c1b73\") " pod="openshift-dns/dns-default-4x886" Jul 08 12:03:47.017792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:47.017753 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5pzr4\"" Jul 08 12:03:47.025591 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:47.025552 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4x886" Jul 08 12:03:48.039780 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:48.039745 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4x886"] Jul 08 12:03:48.043668 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:48.043638 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd593b87b_d888_4119_a222_83aebe7c1b73.slice/crio-225f1c7a06d234934cfc81f927f6aacf15d7ebb72ba35b6b8146d3717be75380 WatchSource:0}: Error finding container 225f1c7a06d234934cfc81f927f6aacf15d7ebb72ba35b6b8146d3717be75380: Status 404 returned error can't find the container with id 225f1c7a06d234934cfc81f927f6aacf15d7ebb72ba35b6b8146d3717be75380 Jul 08 12:03:48.256947 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:48.256832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4x886" event={"ID":"d593b87b-d888-4119-a222-83aebe7c1b73","Type":"ContainerStarted","Data":"225f1c7a06d234934cfc81f927f6aacf15d7ebb72ba35b6b8146d3717be75380"} Jul 08 12:03:48.258323 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:48.258297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4sn6j" event={"ID":"eb192faa-37da-4e22-90dc-613baf99d463","Type":"ContainerStarted","Data":"c4b7045b386c78bdce50b12978ba828b2b0da87b8fdd9e40a771c1c3d04e5d7d"} Jul 08 12:03:49.263380 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:49.263344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4sn6j" event={"ID":"eb192faa-37da-4e22-90dc-613baf99d463","Type":"ContainerStarted","Data":"ac2513a58d051c922bcf35a7de692b7f28c761f8b390905c43208b0083b9743b"} Jul 08 12:03:51.164382 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:51.164343 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rnpfb" Jul 08 12:03:52.276493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:52.276454 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4x886" event={"ID":"d593b87b-d888-4119-a222-83aebe7c1b73","Type":"ContainerStarted","Data":"ad9df250b1f1744710d93f8f8224388c1fad84e09ebdb1e2913dee939a68e161"} Jul 08 12:03:52.276971 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:52.276520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4x886" event={"ID":"d593b87b-d888-4119-a222-83aebe7c1b73","Type":"ContainerStarted","Data":"4468523f09620a6df4c33f2e6a28977ab6b48210dd8fbc0c31843b5a9dfdf022"} Jul 08 12:03:52.276971 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:52.276590 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4x886" Jul 08 12:03:52.278588 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:52.278560 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4sn6j" event={"ID":"eb192faa-37da-4e22-90dc-613baf99d463","Type":"ContainerStarted","Data":"48d810d51d23c409d0b78d587f8ccd00f96f628d974e7549a97e07c35220d1e2"} Jul 08 12:03:52.295628 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:52.295562 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4x886" podStartSLOduration=65.273081832 podStartE2EDuration="1m9.295545888s" totalImagesPullingTime="4.022464056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:52.293709277 +0000 UTC m=+104.976762075" watchObservedRunningTime="2026-07-08 12:03:52.295545888 +0000 UTC m=+104.978598694" Jul 08 12:03:52.314161 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:52.314104 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4sn6j" podStartSLOduration=16.545149105 podStartE2EDuration="24.314089613s" totalImagesPullingTime="7.768940508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:03:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:52.313709032 +0000 UTC m=+104.996761829" watchObservedRunningTime="2026-07-08 12:03:52.314089613 +0000 UTC m=+104.997142410" Jul 08 12:03:54.090336 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.090297 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr"] Jul 08 12:03:54.092599 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.092575 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" Jul 08 12:03:54.095341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.095313 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f8ntf\"" Jul 08 12:03:54.104059 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.104019 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr"] Jul 08 12:03:54.154172 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.154128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqvjw\" (UniqueName: \"kubernetes.io/projected/30487130-6180-4529-b3ae-5728d7856356-kube-api-access-tqvjw\") pod \"network-check-source-989f8b8dc-m2hgr\" (UID: \"30487130-6180-4529-b3ae-5728d7856356\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" Jul 08 12:03:54.219221 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.219182 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d48bf998-dpvjz"] Jul 08 12:03:54.223569 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.223521 2560 patch_prober.go:28] interesting pod/image-registry-5d48bf998-dpvjz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 12:03:54.223789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.223585 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 12:03:54.255528 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.255487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqvjw\" (UniqueName: \"kubernetes.io/projected/30487130-6180-4529-b3ae-5728d7856356-kube-api-access-tqvjw\") pod \"network-check-source-989f8b8dc-m2hgr\" (UID: \"30487130-6180-4529-b3ae-5728d7856356\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" Jul 08 12:03:54.273331 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.273296 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqvjw\" (UniqueName: \"kubernetes.io/projected/30487130-6180-4529-b3ae-5728d7856356-kube-api-access-tqvjw\") pod \"network-check-source-989f8b8dc-m2hgr\" (UID: \"30487130-6180-4529-b3ae-5728d7856356\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" Jul 08 12:03:54.401299 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.401218 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" Jul 08 12:03:54.536122 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:54.536088 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr"] Jul 08 12:03:54.539229 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:03:54.539200 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30487130_6180_4529_b3ae_5728d7856356.slice/crio-eb6098cc2a595b6be2ee8aa7624cb6977c44df64cfbdc91555872c7b354122d8 WatchSource:0}: Error finding container eb6098cc2a595b6be2ee8aa7624cb6977c44df64cfbdc91555872c7b354122d8: Status 404 returned error can't find the container with id eb6098cc2a595b6be2ee8aa7624cb6977c44df64cfbdc91555872c7b354122d8 Jul 08 12:03:55.291940 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:55.291873 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" event={"ID":"30487130-6180-4529-b3ae-5728d7856356","Type":"ContainerStarted","Data":"5a966d458d51461ef737e31dfb339b93467c7be3e9963b003813e55193e66ee1"} Jul 08 12:03:55.292428 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:55.291958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" event={"ID":"30487130-6180-4529-b3ae-5728d7856356","Type":"ContainerStarted","Data":"eb6098cc2a595b6be2ee8aa7624cb6977c44df64cfbdc91555872c7b354122d8"} Jul 08 12:03:55.310365 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:03:55.310294 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m2hgr" podStartSLOduration=1.3102705110000001 podStartE2EDuration="1.310270511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:03:55.30943928 +0000 UTC m=+107.992492079" watchObservedRunningTime="2026-07-08 12:03:55.310270511 +0000 UTC m=+107.993323312" Jul 08 12:04:01.665787 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.665749 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs"] Jul 08 12:04:01.670811 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.670788 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.673240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.673210 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zw2n7\"" Jul 08 12:04:01.673240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.673232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 12:04:01.674494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.674470 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 12:04:01.674494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.674492 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 12:04:01.674674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.674526 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 12:04:01.674674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.674529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 12:04:01.679422 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.679387 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs"] Jul 08 12:04:01.710413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.710365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.710413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.710416 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.710674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.710451 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.710674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.710524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pf84\" (UniqueName: \"kubernetes.io/projected/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-kube-api-access-4pf84\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.811728 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.811691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.811917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.811743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pf84\" (UniqueName: \"kubernetes.io/projected/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-kube-api-access-4pf84\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.811917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.811862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.812030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.811947 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.812090 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:04:01.812035 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 12:04:01.812143 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:04:01.812125 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-tls podName:ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae nodeName:}" failed. No retries permitted until 2026-07-08 12:04:02.312101746 +0000 UTC m=+114.995154530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-npnfs" (UID: "ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae") : secret "prometheus-operator-tls" not found Jul 08 12:04:01.812441 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.812423 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.814613 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.814592 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:01.823806 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:01.823772 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pf84\" (UniqueName: \"kubernetes.io/projected/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-kube-api-access-4pf84\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:02.284789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:02.284756 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4x886" Jul 08 12:04:02.316677 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:02.316643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:02.319289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:02.319253 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-npnfs\" (UID: \"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:02.580659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:02.580565 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" Jul 08 12:04:02.721599 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:02.721564 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs"] Jul 08 12:04:02.724567 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:04:02.724541 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4e2f91_4ba1_40ed_8a25_e5e510dcb0ae.slice/crio-81d5e31b51b2cb1635bbdebc7e0793e61d8bfd67133b4aebcf7d98fa336268b7 WatchSource:0}: Error finding container 81d5e31b51b2cb1635bbdebc7e0793e61d8bfd67133b4aebcf7d98fa336268b7: Status 404 returned error can't find the container with id 81d5e31b51b2cb1635bbdebc7e0793e61d8bfd67133b4aebcf7d98fa336268b7 Jul 08 12:04:03.318512 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:03.318477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" event={"ID":"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae","Type":"ContainerStarted","Data":"81d5e31b51b2cb1635bbdebc7e0793e61d8bfd67133b4aebcf7d98fa336268b7"} Jul 08 12:04:04.223923 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:04.223862 2560 patch_prober.go:28] interesting pod/image-registry-5d48bf998-dpvjz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 12:04:04.224292 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:04.223943 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 12:04:05.327204 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:05.327172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" event={"ID":"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae","Type":"ContainerStarted","Data":"5324ad999f639710a70d4ebf9d29e7e728f4ff76e6a1ca7b253c9bbbeb42ca25"} Jul 08 12:04:06.332381 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:06.332335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" event={"ID":"ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae","Type":"ContainerStarted","Data":"7771b60338a9c240b7dd8a0bc3fe5a460377ac5ca762a5efaaa9880062af1e7c"} Jul 08 12:04:06.358578 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:06.358515 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-npnfs" podStartSLOduration=2.874394225 podStartE2EDuration="5.358497897s" totalImagesPullingTime="2.484103672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:04:06.35786081 +0000 UTC m=+119.040913608" watchObservedRunningTime="2026-07-08 12:04:06.358497897 +0000 UTC m=+119.041550696" Jul 08 12:04:08.071350 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.071311 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6f97b"] Jul 08 12:04:08.075165 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.075135 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.077377 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.077349 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 12:04:08.077666 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.077635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 12:04:08.077814 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.077703 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 12:04:08.077872 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.077812 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pnlbh\"" Jul 08 12:04:08.161185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-wtmp\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161257 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbgnf\" (UniqueName: \"kubernetes.io/projected/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-kube-api-access-zbgnf\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-tls\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-textfile\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-root\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-metrics-client-ca\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.161658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.161542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-sys\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-wtmp\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbgnf\" (UniqueName: \"kubernetes.io/projected/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-kube-api-access-zbgnf\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-tls\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-textfile\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262802 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-wtmp\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.262870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-root\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.263217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.262904 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-root\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.263607 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-metrics-client-ca\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.263668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.263714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-sys\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.263857 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-sys\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:04:08.264246 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:04:08.264432 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-tls podName:d7229dcd-fb1e-4675-8957-f7f91d94aaf0 nodeName:}" failed. No retries permitted until 2026-07-08 12:04:08.764410351 +0000 UTC m=+121.447463129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-tls") pod "node-exporter-6f97b" (UID: "d7229dcd-fb1e-4675-8957-f7f91d94aaf0") : secret "node-exporter-tls" not found Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.264467 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-textfile\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.264475 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-metrics-client-ca\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.266924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.265069 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.273247 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.273146 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.275423 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.275388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbgnf\" (UniqueName: \"kubernetes.io/projected/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-kube-api-access-zbgnf\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.767388 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.767336 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-tls\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.770004 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.769979 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7229dcd-fb1e-4675-8957-f7f91d94aaf0-node-exporter-tls\") pod \"node-exporter-6f97b\" (UID: \"d7229dcd-fb1e-4675-8957-f7f91d94aaf0\") " pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.824689 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.824658 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-xcmhr"] Jul 08 12:04:08.829087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.829064 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:08.831589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.831561 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 12:04:08.831722 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.831642 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6m2cm\"" Jul 08 12:04:08.831901 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.831872 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 12:04:08.838551 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.838519 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-xcmhr"] Jul 08 12:04:08.867754 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.867717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlw6r\" (UniqueName: \"kubernetes.io/projected/95373b22-9f40-43ab-8f9a-e55c26d3abc2-kube-api-access-jlw6r\") pod \"downloads-685bfb5f64-xcmhr\" (UID: \"95373b22-9f40-43ab-8f9a-e55c26d3abc2\") " pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:08.968529 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.968485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlw6r\" (UniqueName: \"kubernetes.io/projected/95373b22-9f40-43ab-8f9a-e55c26d3abc2-kube-api-access-jlw6r\") pod \"downloads-685bfb5f64-xcmhr\" (UID: \"95373b22-9f40-43ab-8f9a-e55c26d3abc2\") " pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:08.986729 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.986690 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6f97b" Jul 08 12:04:08.989579 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:08.989549 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlw6r\" (UniqueName: \"kubernetes.io/projected/95373b22-9f40-43ab-8f9a-e55c26d3abc2-kube-api-access-jlw6r\") pod \"downloads-685bfb5f64-xcmhr\" (UID: \"95373b22-9f40-43ab-8f9a-e55c26d3abc2\") " pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:08.998827 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:04:08.998797 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7229dcd_fb1e_4675_8957_f7f91d94aaf0.slice/crio-7d69b92f3c771e4ec7b5988b351045ee49e26dbf8f1a1d528b6da93291ade214 WatchSource:0}: Error finding container 7d69b92f3c771e4ec7b5988b351045ee49e26dbf8f1a1d528b6da93291ade214: Status 404 returned error can't find the container with id 7d69b92f3c771e4ec7b5988b351045ee49e26dbf8f1a1d528b6da93291ade214 Jul 08 12:04:09.140386 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:09.140283 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:09.296257 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:09.296182 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-xcmhr"] Jul 08 12:04:09.300118 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:04:09.300081 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95373b22_9f40_43ab_8f9a_e55c26d3abc2.slice/crio-80fc9e66106e02424c60665d2b624a12b3f4c9f81339cb13e36d581e0ed5bbb4 WatchSource:0}: Error finding container 80fc9e66106e02424c60665d2b624a12b3f4c9f81339cb13e36d581e0ed5bbb4: Status 404 returned error can't find the container with id 80fc9e66106e02424c60665d2b624a12b3f4c9f81339cb13e36d581e0ed5bbb4 Jul 08 12:04:09.343838 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:09.343801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-xcmhr" event={"ID":"95373b22-9f40-43ab-8f9a-e55c26d3abc2","Type":"ContainerStarted","Data":"80fc9e66106e02424c60665d2b624a12b3f4c9f81339cb13e36d581e0ed5bbb4"} Jul 08 12:04:09.344996 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:09.344970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f97b" event={"ID":"d7229dcd-fb1e-4675-8957-f7f91d94aaf0","Type":"ContainerStarted","Data":"7d69b92f3c771e4ec7b5988b351045ee49e26dbf8f1a1d528b6da93291ade214"} Jul 08 12:04:10.349620 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:10.349579 2560 generic.go:356] "Generic (PLEG): container finished" podID="d7229dcd-fb1e-4675-8957-f7f91d94aaf0" containerID="cd2952a6d79647e758f64d989b0eb6ee99ce57ec4ac5c9c21c9024d6e9cf444e" exitCode=0 Jul 08 12:04:10.350049 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:10.349641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f97b" event={"ID":"d7229dcd-fb1e-4675-8957-f7f91d94aaf0","Type":"ContainerDied","Data":"cd2952a6d79647e758f64d989b0eb6ee99ce57ec4ac5c9c21c9024d6e9cf444e"} Jul 08 12:04:11.356318 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:11.356280 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f97b" event={"ID":"d7229dcd-fb1e-4675-8957-f7f91d94aaf0","Type":"ContainerStarted","Data":"4c5eb3455a86a1021ebbc7568217d43561cc9934a0fd5ff32a0a209305004fe7"} Jul 08 12:04:11.356810 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:11.356347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f97b" event={"ID":"d7229dcd-fb1e-4675-8957-f7f91d94aaf0","Type":"ContainerStarted","Data":"8c14033935cb55576bc28967cf7b675a11b87e39ccca90b897b94f0eb10f14f5"} Jul 08 12:04:11.394143 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:11.394082 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6f97b" podStartSLOduration=2.390389218 podStartE2EDuration="3.394065018s" totalImagesPullingTime="923.791897ms" totalInitContainerRuntime="79.883903ms" isStatefulPod=true podCreationTimestamp="2026-07-08 12:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:04:11.393240779 +0000 UTC m=+124.076293577" watchObservedRunningTime="2026-07-08 12:04:11.394065018 +0000 UTC m=+124.077117816" Jul 08 12:04:12.836125 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.836079 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg"] Jul 08 12:04:12.839456 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.839425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:12.843266 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.843241 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hdr7t\"" Jul 08 12:04:12.843406 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.843241 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 12:04:12.848668 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.848638 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg"] Jul 08 12:04:12.900195 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.900159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2095e034-764b-4d9c-b86d-0a45ac4d9fad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qkbkg\" (UID: \"2095e034-764b-4d9c-b86d-0a45ac4d9fad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:12.900396 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:12.900211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7lqc\" (UniqueName: \"kubernetes.io/projected/2095e034-764b-4d9c-b86d-0a45ac4d9fad-kube-api-access-p7lqc\") pod \"monitoring-plugin-d5dcff7cf-qkbkg\" (UID: \"2095e034-764b-4d9c-b86d-0a45ac4d9fad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:13.001041 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.001004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2095e034-764b-4d9c-b86d-0a45ac4d9fad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qkbkg\" (UID: \"2095e034-764b-4d9c-b86d-0a45ac4d9fad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:13.001241 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.001064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7lqc\" (UniqueName: \"kubernetes.io/projected/2095e034-764b-4d9c-b86d-0a45ac4d9fad-kube-api-access-p7lqc\") pod \"monitoring-plugin-d5dcff7cf-qkbkg\" (UID: \"2095e034-764b-4d9c-b86d-0a45ac4d9fad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:13.003747 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.003712 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2095e034-764b-4d9c-b86d-0a45ac4d9fad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qkbkg\" (UID: \"2095e034-764b-4d9c-b86d-0a45ac4d9fad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:13.011298 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.011267 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7lqc\" (UniqueName: \"kubernetes.io/projected/2095e034-764b-4d9c-b86d-0a45ac4d9fad-kube-api-access-p7lqc\") pod \"monitoring-plugin-d5dcff7cf-qkbkg\" (UID: \"2095e034-764b-4d9c-b86d-0a45ac4d9fad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:13.150422 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.150326 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:13.307722 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.307684 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg"] Jul 08 12:04:13.310312 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:04:13.310271 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2095e034_764b_4d9c_b86d_0a45ac4d9fad.slice/crio-78dcb2df9276517dc92ac17c2e65db1bd061da468f93151212982f5e64ab0457 WatchSource:0}: Error finding container 78dcb2df9276517dc92ac17c2e65db1bd061da468f93151212982f5e64ab0457: Status 404 returned error can't find the container with id 78dcb2df9276517dc92ac17c2e65db1bd061da468f93151212982f5e64ab0457 Jul 08 12:04:13.365487 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:13.365438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" event={"ID":"2095e034-764b-4d9c-b86d-0a45ac4d9fad","Type":"ContainerStarted","Data":"78dcb2df9276517dc92ac17c2e65db1bd061da468f93151212982f5e64ab0457"} Jul 08 12:04:14.228309 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.228060 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:04:14.569286 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.567255 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:04:14.573000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.571665 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.574501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.574360 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 12:04:14.575965 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.575936 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 12:04:14.579604 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579467 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 12:04:14.579604 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579523 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ei9sgeterq8du\"" Jul 08 12:04:14.579604 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579467 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 12:04:14.579604 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579544 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 12:04:14.580002 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579468 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 12:04:14.580002 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579838 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7j82c\"" Jul 08 12:04:14.580002 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579993 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 12:04:14.580375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.579999 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 12:04:14.582790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.580967 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 12:04:14.582790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.581394 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 12:04:14.582790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.581653 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 12:04:14.582790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.581820 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 12:04:14.582790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.582189 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 12:04:14.583185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.583132 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 12:04:14.594520 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.594488 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 12:04:14.594843 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.594818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:04:14.615612 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.615804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.615804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.615804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.615804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615829 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615902 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.615991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616733 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616733 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616633 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616733 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616662 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8pdk\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-kube-api-access-v8pdk\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.616733 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616691 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.617038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616748 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.617038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.617038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616838 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.617038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.617038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.617038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.616985 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718056 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718056 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718149 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718177 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pdk\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-kube-api-access-v8pdk\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718562 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.718852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.718629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.722329 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.719419 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.722329 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.719719 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.722329 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.720245 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.722329 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.721551 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.722679 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.722340 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.723063 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.723031 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.723825 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.723794 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.724524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.724474 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.727494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.727430 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.727494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.727478 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.730336 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.730250 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.730497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.730436 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.730577 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.730550 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.731490 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.731048 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.731490 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.731239 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.731490 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.731472 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.733002 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.732973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.733528 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.733478 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.736061 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.736012 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.743547 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.743509 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pdk\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-kube-api-access-v8pdk\") pod \"prometheus-k8s-0\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:14.896750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:14.896660 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:15.276592 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:15.276471 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:04:15.280719 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:04:15.280683 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d559c31_612c_44cb_9c76_f3a7ca43c8c5.slice/crio-044ca2368adda35a23189b849490c376aa94d955fc6ad409c74b97255b30d30b WatchSource:0}: Error finding container 044ca2368adda35a23189b849490c376aa94d955fc6ad409c74b97255b30d30b: Status 404 returned error can't find the container with id 044ca2368adda35a23189b849490c376aa94d955fc6ad409c74b97255b30d30b Jul 08 12:04:15.377939 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:15.377878 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" event={"ID":"2095e034-764b-4d9c-b86d-0a45ac4d9fad","Type":"ContainerStarted","Data":"b760cb323a34bd1fd1b402da6d79227442eb323d872940b7df7c3711c0c76f41"} Jul 08 12:04:15.378132 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:15.377961 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:15.379278 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:15.379242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"044ca2368adda35a23189b849490c376aa94d955fc6ad409c74b97255b30d30b"} Jul 08 12:04:15.384309 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:15.384278 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" Jul 08 12:04:15.397064 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:15.397002 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qkbkg" podStartSLOduration=1.57032254 podStartE2EDuration="3.396986233s" totalImagesPullingTime="1.826663693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:04:15.395960058 +0000 UTC m=+128.079012857" watchObservedRunningTime="2026-07-08 12:04:15.396986233 +0000 UTC m=+128.080039010" Jul 08 12:04:17.390449 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.390406 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" exitCode=0 Jul 08 12:04:17.390992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.390485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} Jul 08 12:04:17.547466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.547418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:04:17.550565 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.550491 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2c61-626f-4c5b-bcba-91ec3c1073d4-metrics-certs\") pod \"network-metrics-daemon-f8lqw\" (UID: \"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4\") " pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:04:17.604071 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.604038 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rdkjt\"" Jul 08 12:04:17.611384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.611353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8lqw" Jul 08 12:04:17.774277 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:17.774248 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f8lqw"] Jul 08 12:04:17.777624 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:04:17.777195 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5e2c61_626f_4c5b_bcba_91ec3c1073d4.slice/crio-6dece63e74d987b583a1a8e2cc60481db943fcdcde1810d9abffce7bde123eab WatchSource:0}: Error finding container 6dece63e74d987b583a1a8e2cc60481db943fcdcde1810d9abffce7bde123eab: Status 404 returned error can't find the container with id 6dece63e74d987b583a1a8e2cc60481db943fcdcde1810d9abffce7bde123eab Jul 08 12:04:18.396966 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:18.396926 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f8lqw" event={"ID":"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4","Type":"ContainerStarted","Data":"6dece63e74d987b583a1a8e2cc60481db943fcdcde1810d9abffce7bde123eab"} Jul 08 12:04:19.240521 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.240470 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerName="registry" containerID="cri-o://6d2a35c5705d52c0418714dd02ddc6bc38e76ce5e31bd19dc1c684f299aa6ae7" gracePeriod=30 Jul 08 12:04:19.404817 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.404586 2560 generic.go:356] "Generic (PLEG): container finished" podID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerID="6d2a35c5705d52c0418714dd02ddc6bc38e76ce5e31bd19dc1c684f299aa6ae7" exitCode=0 Jul 08 12:04:19.404817 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.404630 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" event={"ID":"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e","Type":"ContainerDied","Data":"6d2a35c5705d52c0418714dd02ddc6bc38e76ce5e31bd19dc1c684f299aa6ae7"} Jul 08 12:04:19.602774 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.602711 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:04:19.667307 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667269 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-ca-trust-extracted\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667324 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667386 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-certificates\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667424 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-image-registry-private-configuration\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667496 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-trusted-ca\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667531 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48pk5\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-kube-api-access-48pk5\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667613 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-bound-sa-token\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.667719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.667667 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-installation-pull-secrets\") pod \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\" (UID: \"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e\") " Jul 08 12:04:19.671259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.671165 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:04:19.671694 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.671649 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:04:19.677118 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.677050 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:04:19.677313 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.677269 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:04:19.678021 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.677991 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:04:19.682617 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.682562 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-kube-api-access-48pk5" (OuterVolumeSpecName: "kube-api-access-48pk5") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "kube-api-access-48pk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:04:19.683360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.683324 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:04:19.684016 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.683973 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" (UID: "cfd28cfb-6eb0-491a-bf23-1b760b0cc25e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:04:19.768647 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768599 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-certificates\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768647 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768645 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-image-registry-private-configuration\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768662 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-trusted-ca\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768676 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48pk5\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-kube-api-access-48pk5\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768690 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-bound-sa-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768703 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-installation-pull-secrets\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768716 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-ca-trust-extracted\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:19.768931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:19.768729 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e-registry-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:04:20.413267 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.413217 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f8lqw" event={"ID":"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4","Type":"ContainerStarted","Data":"252cadae64b9fda06745fae71cd42be694aa1dba4e95a6c7c760dd948008ec82"} Jul 08 12:04:20.413737 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.413301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f8lqw" event={"ID":"cd5e2c61-626f-4c5b-bcba-91ec3c1073d4","Type":"ContainerStarted","Data":"d2d896e5f14e4dcaae8ec2134dc83a10304ae86e812b375266c473c5feda6a0b"} Jul 08 12:04:20.415625 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.415583 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" Jul 08 12:04:20.415953 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.415583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d48bf998-dpvjz" event={"ID":"cfd28cfb-6eb0-491a-bf23-1b760b0cc25e","Type":"ContainerDied","Data":"6bb7ca8b733eff169136de24a629d4ff8e8aa2dd74e8150766559ec1432cadb7"} Jul 08 12:04:20.416056 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.416006 2560 scope.go:117] "RemoveContainer" containerID="6d2a35c5705d52c0418714dd02ddc6bc38e76ce5e31bd19dc1c684f299aa6ae7" Jul 08 12:04:20.434219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.434152 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f8lqw" podStartSLOduration=130.844048949 podStartE2EDuration="2m12.434128989s" totalImagesPullingTime="1.59008004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:04:20.43263419 +0000 UTC m=+133.115686988" watchObservedRunningTime="2026-07-08 12:04:20.434128989 +0000 UTC m=+133.117181788" Jul 08 12:04:20.450963 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.450924 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d48bf998-dpvjz"] Jul 08 12:04:20.463309 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:20.463260 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d48bf998-dpvjz"] Jul 08 12:04:21.887132 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:21.887091 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" path="/var/lib/kubelet/pods/cfd28cfb-6eb0-491a-bf23-1b760b0cc25e/volumes" Jul 08 12:04:22.426744 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:22.426707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} Jul 08 12:04:22.426980 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:22.426793 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} Jul 08 12:04:31.468556 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.468453 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-xcmhr" event={"ID":"95373b22-9f40-43ab-8f9a-e55c26d3abc2","Type":"ContainerStarted","Data":"e49e5596631978a24c7c3ec814e77c2de5b304c9378eac2371765e8c8863c83f"} Jul 08 12:04:31.469066 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.468941 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:31.473010 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.472980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} Jul 08 12:04:31.473178 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.473088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} Jul 08 12:04:31.473178 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.473151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} Jul 08 12:04:31.473254 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.473191 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerStarted","Data":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} Jul 08 12:04:31.481482 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.481442 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-xcmhr" Jul 08 12:04:31.489865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.489803 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-xcmhr" podStartSLOduration=1.809993284 podStartE2EDuration="23.489780501s" totalImagesPullingTime="21.679787217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:04:31.486994866 +0000 UTC m=+144.170047665" watchObservedRunningTime="2026-07-08 12:04:31.489780501 +0000 UTC m=+144.172833298" Jul 08 12:04:31.521321 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:31.521246 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.822694054 podStartE2EDuration="17.521221517s" totalImagesPullingTime="14.669441794s" totalInitContainerRuntime="29.085669ms" isStatefulPod=false podCreationTimestamp="2026-07-08 12:04:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:04:31.51961673 +0000 UTC m=+144.202669529" watchObservedRunningTime="2026-07-08 12:04:31.521221517 +0000 UTC m=+144.204274318" Jul 08 12:04:34.897637 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:34.897591 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:04:57.578267 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:57.578228 2560 generic.go:356] "Generic (PLEG): container finished" podID="85462481-e494-40d8-b1d0-959c40dce0b7" containerID="bc801929bb793f4be7bde9529162e34143b663c682097531da186764165e3d90" exitCode=0 Jul 08 12:04:57.578694 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:57.578304 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" event={"ID":"85462481-e494-40d8-b1d0-959c40dce0b7","Type":"ContainerDied","Data":"bc801929bb793f4be7bde9529162e34143b663c682097531da186764165e3d90"} Jul 08 12:04:57.578694 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:57.578670 2560 scope.go:117] "RemoveContainer" containerID="bc801929bb793f4be7bde9529162e34143b663c682097531da186764165e3d90" Jul 08 12:04:58.584090 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:04:58.584049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-glfxg" event={"ID":"85462481-e494-40d8-b1d0-959c40dce0b7","Type":"ContainerStarted","Data":"cc1010d7a548c89521c38b219d1519905e5b8dd28af1c89ecdaca5fb7c25fb1e"} Jul 08 12:05:14.897554 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:14.897504 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:14.976851 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:14.976821 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:15.664972 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:15.664944 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:32.977212 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977166 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:05:32.977674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977570 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy" containerID="cri-o://1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" gracePeriod=600 Jul 08 12:05:32.977674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977596 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="thanos-sidecar" containerID="cri-o://96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" gracePeriod=600 Jul 08 12:05:32.977674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977623 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="config-reloader" containerID="cri-o://4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" gracePeriod=600 Jul 08 12:05:32.977674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977620 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-web" containerID="cri-o://d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" gracePeriod=600 Jul 08 12:05:32.977926 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977551 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="prometheus" containerID="cri-o://c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" gracePeriod=600 Jul 08 12:05:32.977926 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:32.977743 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" gracePeriod=600 Jul 08 12:05:33.243987 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.243956 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.409925 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.409830 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-kubelet-serving-ca-bundle\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.409925 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.409939 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-thanos-prometheus-http-client-file\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410190 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.409967 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-metrics-client-ca\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410190 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.409986 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-kube-rbac-proxy\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410190 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410008 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-grpc-tls\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410190 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410024 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-1\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410190 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410046 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-db\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410205 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410246 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-tls-assets\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410277 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-2\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410272 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410303 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-tls\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410339 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410374 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410404 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-web-config\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410444 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8pdk\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-kube-api-access-v8pdk\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410478 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config-out\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410476 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410511 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-serving-certs-ca-bundle\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410537 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-metrics-client-certs\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410583 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-0\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410609 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-trusted-ca-bundle\") pod \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\" (UID: \"9d559c31-612c-44cb-9c76-f3a7ca43c8c5\") " Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410632 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410846 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410862 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-metrics-client-ca\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.410928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.410876 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.411624 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.411069 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:05:33.411624 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.411211 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.413375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.413339 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:05:33.413375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.413353 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.413596 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.413386 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.413596 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.413572 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.413835 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.413810 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.413929 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.413862 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.414038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.414014 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.414965 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.414776 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.414965 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.414937 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config" (OuterVolumeSpecName: "config") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.415492 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.415464 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:05:33.415782 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.415761 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-kube-api-access-v8pdk" (OuterVolumeSpecName: "kube-api-access-v8pdk") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "kube-api-access-v8pdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:05:33.415944 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.415926 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.416517 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.416490 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.416607 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.416490 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config-out" (OuterVolumeSpecName: "config-out") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:05:33.425634 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.425587 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-web-config" (OuterVolumeSpecName: "web-config") pod "9d559c31-612c-44cb-9c76-f3a7ca43c8c5" (UID: "9d559c31-612c-44cb-9c76-f3a7ca43c8c5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 12:05:33.511281 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511247 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-kube-rbac-proxy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511281 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511281 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-grpc-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511291 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-db\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511300 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511308 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-tls-assets\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511317 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511325 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511333 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511345 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511356 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-web-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511364 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8pdk\" (UniqueName: \"kubernetes.io/projected/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-kube-api-access-v8pdk\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511372 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-config-out\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511380 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511388 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-secret-metrics-client-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511396 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511404 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.511468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.511412 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d559c31-612c-44cb-9c76-f3a7ca43c8c5-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:05:33.716316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716281 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" exitCode=0 Jul 08 12:05:33.716316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716308 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" exitCode=0 Jul 08 12:05:33.716316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716317 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" exitCode=0 Jul 08 12:05:33.716316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716325 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" exitCode=0 Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716332 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" exitCode=0 Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716337 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" exitCode=0 Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716372 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716369 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716502 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} Jul 08 12:05:33.716589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716567 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.716912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716592 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} Jul 08 12:05:33.716912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716638 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} Jul 08 12:05:33.716912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716679 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} Jul 08 12:05:33.716912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.716721 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d559c31-612c-44cb-9c76-f3a7ca43c8c5","Type":"ContainerDied","Data":"044ca2368adda35a23189b849490c376aa94d955fc6ad409c74b97255b30d30b"} Jul 08 12:05:33.725100 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.725077 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.732775 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.732754 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.741114 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.741090 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.743500 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.743475 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:05:33.747761 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.747736 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:05:33.749861 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.749840 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.757981 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.757959 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.766647 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.766622 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.774360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.774339 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.774710 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.774684 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.774802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.774725 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} err="failed to get container status \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" Jul 08 12:05:33.774802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.774775 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.775166 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.775131 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.775262 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.775166 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} err="failed to get container status \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" Jul 08 12:05:33.775262 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.775187 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.775482 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.775453 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.775564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.775487 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} err="failed to get container status \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" Jul 08 12:05:33.775564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.775508 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.775779 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.775761 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.775831 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.775785 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} err="failed to get container status \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" Jul 08 12:05:33.775831 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.775801 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.776148 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.776129 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.776226 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776154 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} err="failed to get container status \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" Jul 08 12:05:33.776226 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776180 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.776469 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.776452 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.776519 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776474 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} err="failed to get container status \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" Jul 08 12:05:33.776519 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776501 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.776681 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776663 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:05:33.776778 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:05:33.776761 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.776820 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776783 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} err="failed to get container status \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" Jul 08 12:05:33.776820 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776799 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.776955 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776941 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy" Jul 08 12:05:33.777000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776961 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy" Jul 08 12:05:33.777000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776977 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="init-config-reloader" Jul 08 12:05:33.777000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776984 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="init-config-reloader" Jul 08 12:05:33.777000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.776993 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="thanos-sidecar" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777002 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="thanos-sidecar" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777013 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="config-reloader" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777022 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="config-reloader" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777032 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerName="registry" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777038 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerName="registry" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777045 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-web" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777050 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-web" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777057 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-thanos" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777063 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-thanos" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777057 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} err="failed to get container status \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777073 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="prometheus" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777078 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="prometheus" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777080 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777117 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="config-reloader" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777126 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="thanos-sidecar" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777132 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd28cfb-6eb0-491a-bf23-1b760b0cc25e" containerName="registry" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777137 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-web" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777142 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777149 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="prometheus" Jul 08 12:05:33.777171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777154 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" containerName="kube-rbac-proxy-thanos" Jul 08 12:05:33.777801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777304 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} err="failed to get container status \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" Jul 08 12:05:33.777801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777332 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.777801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777574 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} err="failed to get container status \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" Jul 08 12:05:33.777801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777590 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.777801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777785 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} err="failed to get container status \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" Jul 08 12:05:33.777801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.777798 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.778062 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778044 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} err="failed to get container status \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" Jul 08 12:05:33.778102 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778062 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.778251 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778236 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} err="failed to get container status \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" Jul 08 12:05:33.778304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778250 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.778507 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778487 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} err="failed to get container status \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" Jul 08 12:05:33.778591 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778511 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.778822 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778795 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} err="failed to get container status \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" Jul 08 12:05:33.778906 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.778822 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.779114 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779097 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} err="failed to get container status \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" Jul 08 12:05:33.779161 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779115 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.779339 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779323 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} err="failed to get container status \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" Jul 08 12:05:33.779382 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779349 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.779581 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779561 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} err="failed to get container status \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" Jul 08 12:05:33.779702 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779582 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.779836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779818 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} err="failed to get container status \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" Jul 08 12:05:33.779928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.779839 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.780143 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.780124 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} err="failed to get container status \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" Jul 08 12:05:33.780219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.780144 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.782458 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.782421 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} err="failed to get container status \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" Jul 08 12:05:33.782458 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.782448 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.782628 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.782473 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.782771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.782743 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} err="failed to get container status \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" Jul 08 12:05:33.782833 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.782773 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.783465 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.783435 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} err="failed to get container status \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" Jul 08 12:05:33.783556 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.783467 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.783768 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.783741 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} err="failed to get container status \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" Jul 08 12:05:33.783768 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.783768 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.784075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784053 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} err="failed to get container status \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" Jul 08 12:05:33.784133 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784076 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.784339 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784313 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} err="failed to get container status \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" Jul 08 12:05:33.784391 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784343 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.784624 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784605 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} err="failed to get container status \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" Jul 08 12:05:33.784686 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784627 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.784858 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784839 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} err="failed to get container status \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" Jul 08 12:05:33.784963 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.784860 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.785186 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785165 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} err="failed to get container status \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" Jul 08 12:05:33.785186 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785186 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.785315 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785295 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ei9sgeterq8du\"" Jul 08 12:05:33.785484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785460 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} err="failed to get container status \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" Jul 08 12:05:33.785547 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785482 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 12:05:33.785547 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785488 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.785804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785786 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} err="failed to get container status \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" Jul 08 12:05:33.785851 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.785806 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.786061 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786041 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 12:05:33.786146 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786088 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} err="failed to get container status \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" Jul 08 12:05:33.786146 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786108 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.786395 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786336 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 12:05:33.786395 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786357 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 12:05:33.786582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786417 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7j82c\"" Jul 08 12:05:33.786582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786491 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} err="failed to get container status \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" Jul 08 12:05:33.786582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786513 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.786582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786544 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 12:05:33.786849 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786612 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 12:05:33.786849 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786622 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 12:05:33.786849 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786762 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} err="failed to get container status \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" Jul 08 12:05:33.786849 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.786784 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787111 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787127 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} err="failed to get container status \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787156 2560 scope.go:117] "RemoveContainer" containerID="e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787213 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787222 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 12:05:33.787466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787364 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 12:05:33.787914 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787609 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1"} err="failed to get container status \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": rpc error: code = NotFound desc = could not find container \"e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1\": container with ID starting with e0f7f8e72057574ec17b431924040dc2c13b7d1ab8461cf787c837f2845069a1 not found: ID does not exist" Jul 08 12:05:33.787914 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787633 2560 scope.go:117] "RemoveContainer" containerID="1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db" Jul 08 12:05:33.787914 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.787643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 12:05:33.788102 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.788053 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db"} err="failed to get container status \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": rpc error: code = NotFound desc = could not find container \"1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db\": container with ID starting with 1b8c0de44007cb8fd7b3d01bf8ffee656589d4d2f9a69cdcb2c40244140e45db not found: ID does not exist" Jul 08 12:05:33.788102 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.788079 2560 scope.go:117] "RemoveContainer" containerID="d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0" Jul 08 12:05:33.788383 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.788356 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0"} err="failed to get container status \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": rpc error: code = NotFound desc = could not find container \"d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0\": container with ID starting with d914ae936446a9e8d00175d0f51dd6c7d48faf50e3d85b843c772e0a9a8974f0 not found: ID does not exist" Jul 08 12:05:33.788475 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.788387 2560 scope.go:117] "RemoveContainer" containerID="96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf" Jul 08 12:05:33.788831 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.788806 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf"} err="failed to get container status \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": rpc error: code = NotFound desc = could not find container \"96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf\": container with ID starting with 96ca6e4e2c5b70c601db158e7b498add959f675214f5dc0cdd6d15662bd8adaf not found: ID does not exist" Jul 08 12:05:33.788935 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.788832 2560 scope.go:117] "RemoveContainer" containerID="4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57" Jul 08 12:05:33.789109 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.789082 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57"} err="failed to get container status \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": rpc error: code = NotFound desc = could not find container \"4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57\": container with ID starting with 4b6aefaf13830a8589437b2d83fab83fbdf70257435a58b6314fb04031894d57 not found: ID does not exist" Jul 08 12:05:33.789177 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.789108 2560 scope.go:117] "RemoveContainer" containerID="c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc" Jul 08 12:05:33.789377 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.789354 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc"} err="failed to get container status \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": rpc error: code = NotFound desc = could not find container \"c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc\": container with ID starting with c195894e7ce4e380472116db9a74a0280997d8006613aff956a190309af1c3dc not found: ID does not exist" Jul 08 12:05:33.789425 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.789384 2560 scope.go:117] "RemoveContainer" containerID="08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4" Jul 08 12:05:33.789706 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.789681 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4"} err="failed to get container status \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": rpc error: code = NotFound desc = could not find container \"08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4\": container with ID starting with 08bc0cee04f1c39659aaa851be03279a09f468cf41ffa2e9092466a26a9de8d4 not found: ID does not exist" Jul 08 12:05:33.791584 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.791563 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 12:05:33.792305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.792284 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 12:05:33.799726 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.799699 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:05:33.885953 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.885915 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d559c31-612c-44cb-9c76-f3a7ca43c8c5" path="/var/lib/kubelet/pods/9d559c31-612c-44cb-9c76-f3a7ca43c8c5/volumes" Jul 08 12:05:33.914324 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-config\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-web-config\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914591 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b95f60a5-7097-4596-bb75-304d94d78272-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914680 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b95f60a5-7097-4596-bb75-304d94d78272-config-out\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f59s5\" (UniqueName: \"kubernetes.io/projected/b95f60a5-7097-4596-bb75-304d94d78272-kube-api-access-f59s5\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.914808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:33.915061 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:33.914824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016145 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b95f60a5-7097-4596-bb75-304d94d78272-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b95f60a5-7097-4596-bb75-304d94d78272-config-out\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f59s5\" (UniqueName: \"kubernetes.io/projected/b95f60a5-7097-4596-bb75-304d94d78272-kube-api-access-f59s5\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016433 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-config\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-web-config\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.016750 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.017464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.017464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.017464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.017464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.016862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.017464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.017308 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.017464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.017380 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.018804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.018768 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.019322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.019292 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.019556 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.019526 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.019780 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.019750 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b95f60a5-7097-4596-bb75-304d94d78272-config-out\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.019856 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.019822 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.020135 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.020089 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.020914 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.020836 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.021630 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.021562 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-web-config\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.021630 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.021565 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b95f60a5-7097-4596-bb75-304d94d78272-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.021630 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.021626 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.022064 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.021984 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.022214 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.022187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-config\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.022446 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.022421 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b95f60a5-7097-4596-bb75-304d94d78272-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.022561 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.022538 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.023099 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.023079 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.023183 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.023106 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.023740 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.023719 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b95f60a5-7097-4596-bb75-304d94d78272-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.029096 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.029062 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f59s5\" (UniqueName: \"kubernetes.io/projected/b95f60a5-7097-4596-bb75-304d94d78272-kube-api-access-f59s5\") pod \"prometheus-k8s-0\" (UID: \"b95f60a5-7097-4596-bb75-304d94d78272\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.094880 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.094834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:05:34.250059 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.250023 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 12:05:34.253101 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:05:34.253067 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95f60a5_7097_4596_bb75_304d94d78272.slice/crio-75c700aa70961bd08bf2d005ff73d57a8297d6707ec2e0d19e6629e4907d11b8 WatchSource:0}: Error finding container 75c700aa70961bd08bf2d005ff73d57a8297d6707ec2e0d19e6629e4907d11b8: Status 404 returned error can't find the container with id 75c700aa70961bd08bf2d005ff73d57a8297d6707ec2e0d19e6629e4907d11b8 Jul 08 12:05:34.722345 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.722245 2560 generic.go:356] "Generic (PLEG): container finished" podID="b95f60a5-7097-4596-bb75-304d94d78272" containerID="7a94d2a1b887b82ae390279e15d1f0f44e411cbe7ba6bf219d37212d10066dd8" exitCode=0 Jul 08 12:05:34.722345 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.722293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerDied","Data":"7a94d2a1b887b82ae390279e15d1f0f44e411cbe7ba6bf219d37212d10066dd8"} Jul 08 12:05:34.722591 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:34.722351 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"75c700aa70961bd08bf2d005ff73d57a8297d6707ec2e0d19e6629e4907d11b8"} Jul 08 12:05:35.730167 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.730129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"328d3ca2ea2ebc2b8417854f768bff4ca874daebc8a0121c1d2a6f4000fb837a"} Jul 08 12:05:35.730564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.730229 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"20f8d5569e3c47092de0f506a9e574cce7b2b60c7f998e8bdd284b4eeb76d9b3"} Jul 08 12:05:35.730564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.730257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"36b49ba8e55d1a71e06e8a10e8945a4f8e407aa337b5a5a11ae79a6c26336cbc"} Jul 08 12:05:35.730564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.730281 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"2f512bd77443ecfcbd15588548ed911f372e27e046694663af61f382ec4583a4"} Jul 08 12:05:35.730564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.730306 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"1927b74f5ae64c1972f2e2f8158da300155fbd43c628311857a547bbf5712901"} Jul 08 12:05:35.730564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.730330 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b95f60a5-7097-4596-bb75-304d94d78272","Type":"ContainerStarted","Data":"713fd4d76a9e9bfab896d0634670f9338ecf6ac45c4171870b14b3fd34a94c1a"} Jul 08 12:05:35.763757 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:35.763647 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7436044710000003 podStartE2EDuration="2.763622394s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.017923ms" isStatefulPod=false podCreationTimestamp="2026-07-08 12:05:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:05:35.762619195 +0000 UTC m=+208.445671995" watchObservedRunningTime="2026-07-08 12:05:35.763622394 +0000 UTC m=+208.446675194" Jul 08 12:05:39.095912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:05:39.095849 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:06:34.095348 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:34.095244 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:06:34.112419 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:34.112383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:06:34.968260 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:34.968228 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 12:06:48.718773 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.718735 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm"] Jul 08 12:06:48.721278 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.721257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:48.723927 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.723897 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-46z8p\"" Jul 08 12:06:48.724089 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.723936 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 12:06:48.724089 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.724071 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 12:06:48.735605 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.735572 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm"] Jul 08 12:06:48.905911 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.905843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4789014a-ef2f-4a3d-ab92-5c86eab9be2b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4cwxm\" (UID: \"4789014a-ef2f-4a3d-ab92-5c86eab9be2b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:48.906094 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:48.905994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7clqd\" (UniqueName: \"kubernetes.io/projected/4789014a-ef2f-4a3d-ab92-5c86eab9be2b-kube-api-access-7clqd\") pod \"cert-manager-operator-controller-manager-55c444bf86-4cwxm\" (UID: \"4789014a-ef2f-4a3d-ab92-5c86eab9be2b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:49.007470 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:49.007361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4789014a-ef2f-4a3d-ab92-5c86eab9be2b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4cwxm\" (UID: \"4789014a-ef2f-4a3d-ab92-5c86eab9be2b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:49.007470 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:49.007448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7clqd\" (UniqueName: \"kubernetes.io/projected/4789014a-ef2f-4a3d-ab92-5c86eab9be2b-kube-api-access-7clqd\") pod \"cert-manager-operator-controller-manager-55c444bf86-4cwxm\" (UID: \"4789014a-ef2f-4a3d-ab92-5c86eab9be2b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:49.007786 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:49.007767 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4789014a-ef2f-4a3d-ab92-5c86eab9be2b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4cwxm\" (UID: \"4789014a-ef2f-4a3d-ab92-5c86eab9be2b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:49.022508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:49.022472 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7clqd\" (UniqueName: \"kubernetes.io/projected/4789014a-ef2f-4a3d-ab92-5c86eab9be2b-kube-api-access-7clqd\") pod \"cert-manager-operator-controller-manager-55c444bf86-4cwxm\" (UID: \"4789014a-ef2f-4a3d-ab92-5c86eab9be2b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:49.031122 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:49.031095 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" Jul 08 12:06:49.182000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:49.181974 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm"] Jul 08 12:06:49.184875 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:06:49.184842 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4789014a_ef2f_4a3d_ab92_5c86eab9be2b.slice/crio-f8ca82d761c0c575a1c7f08259ded106abcfed9351ccaaf90b6ef46ba792d88d WatchSource:0}: Error finding container f8ca82d761c0c575a1c7f08259ded106abcfed9351ccaaf90b6ef46ba792d88d: Status 404 returned error can't find the container with id f8ca82d761c0c575a1c7f08259ded106abcfed9351ccaaf90b6ef46ba792d88d Jul 08 12:06:50.009858 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:50.009819 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" event={"ID":"4789014a-ef2f-4a3d-ab92-5c86eab9be2b","Type":"ContainerStarted","Data":"f8ca82d761c0c575a1c7f08259ded106abcfed9351ccaaf90b6ef46ba792d88d"} Jul 08 12:06:52.021215 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:52.021177 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" event={"ID":"4789014a-ef2f-4a3d-ab92-5c86eab9be2b","Type":"ContainerStarted","Data":"9527ec93d6520977dc669f4008dbbb8dfe97186304e7e70e28812e6722a4d5c6"} Jul 08 12:06:52.050274 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:52.050218 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4cwxm" podStartSLOduration=1.4232502980000001 podStartE2EDuration="4.050200436s" totalImagesPullingTime="2.626950138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:06:52.048528378 +0000 UTC m=+284.731581176" watchObservedRunningTime="2026-07-08 12:06:52.050200436 +0000 UTC m=+284.733253234" Jul 08 12:06:55.290526 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.290486 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h4pds"] Jul 08 12:06:55.293071 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.293051 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.298532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.297019 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 12:06:55.298532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.297261 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 12:06:55.298532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.298082 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5rq6n\"" Jul 08 12:06:55.310861 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.310814 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h4pds"] Jul 08 12:06:55.334766 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.334728 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s4shh"] Jul 08 12:06:55.337161 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.337138 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.340019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.339991 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qvglc\"" Jul 08 12:06:55.365591 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.365554 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s4shh"] Jul 08 12:06:55.459589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.459542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb21233e-f257-4fd9-9ab7-d53e72d3fda8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h4pds\" (UID: \"bb21233e-f257-4fd9-9ab7-d53e72d3fda8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.459762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.459598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5971526-bac8-4a3e-9894-fc870ff69081-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s4shh\" (UID: \"b5971526-bac8-4a3e-9894-fc870ff69081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.459762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.459630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn5ww\" (UniqueName: \"kubernetes.io/projected/b5971526-bac8-4a3e-9894-fc870ff69081-kube-api-access-mn5ww\") pod \"cert-manager-cainjector-559cdd8c87-s4shh\" (UID: \"b5971526-bac8-4a3e-9894-fc870ff69081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.459762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.459651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9bss\" (UniqueName: \"kubernetes.io/projected/bb21233e-f257-4fd9-9ab7-d53e72d3fda8-kube-api-access-h9bss\") pod \"cert-manager-webhook-6486ffd997-h4pds\" (UID: \"bb21233e-f257-4fd9-9ab7-d53e72d3fda8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.560742 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.560655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb21233e-f257-4fd9-9ab7-d53e72d3fda8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h4pds\" (UID: \"bb21233e-f257-4fd9-9ab7-d53e72d3fda8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.560742 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.560713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5971526-bac8-4a3e-9894-fc870ff69081-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s4shh\" (UID: \"b5971526-bac8-4a3e-9894-fc870ff69081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.560957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.560825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn5ww\" (UniqueName: \"kubernetes.io/projected/b5971526-bac8-4a3e-9894-fc870ff69081-kube-api-access-mn5ww\") pod \"cert-manager-cainjector-559cdd8c87-s4shh\" (UID: \"b5971526-bac8-4a3e-9894-fc870ff69081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.560957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.560864 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9bss\" (UniqueName: \"kubernetes.io/projected/bb21233e-f257-4fd9-9ab7-d53e72d3fda8-kube-api-access-h9bss\") pod \"cert-manager-webhook-6486ffd997-h4pds\" (UID: \"bb21233e-f257-4fd9-9ab7-d53e72d3fda8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.585274 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.585242 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9bss\" (UniqueName: \"kubernetes.io/projected/bb21233e-f257-4fd9-9ab7-d53e72d3fda8-kube-api-access-h9bss\") pod \"cert-manager-webhook-6486ffd997-h4pds\" (UID: \"bb21233e-f257-4fd9-9ab7-d53e72d3fda8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.585428 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.585314 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5971526-bac8-4a3e-9894-fc870ff69081-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s4shh\" (UID: \"b5971526-bac8-4a3e-9894-fc870ff69081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.585529 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.585501 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn5ww\" (UniqueName: \"kubernetes.io/projected/b5971526-bac8-4a3e-9894-fc870ff69081-kube-api-access-mn5ww\") pod \"cert-manager-cainjector-559cdd8c87-s4shh\" (UID: \"b5971526-bac8-4a3e-9894-fc870ff69081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.585668 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.585646 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb21233e-f257-4fd9-9ab7-d53e72d3fda8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h4pds\" (UID: \"bb21233e-f257-4fd9-9ab7-d53e72d3fda8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.623469 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.623433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:55.648267 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.648234 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" Jul 08 12:06:55.812278 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.812085 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h4pds"] Jul 08 12:06:55.816532 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:06:55.816491 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb21233e_f257_4fd9_9ab7_d53e72d3fda8.slice/crio-59f380ec7247d1829f4f784890299bbee323e815e1085926936d9422a8574993 WatchSource:0}: Error finding container 59f380ec7247d1829f4f784890299bbee323e815e1085926936d9422a8574993: Status 404 returned error can't find the container with id 59f380ec7247d1829f4f784890299bbee323e815e1085926936d9422a8574993 Jul 08 12:06:55.824413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:55.824384 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s4shh"] Jul 08 12:06:55.827160 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:06:55.827128 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5971526_bac8_4a3e_9894_fc870ff69081.slice/crio-593946c3ab596ea5f8b3327cb130763d39c95f3e5dd0ca73d21413d5110cbf53 WatchSource:0}: Error finding container 593946c3ab596ea5f8b3327cb130763d39c95f3e5dd0ca73d21413d5110cbf53: Status 404 returned error can't find the container with id 593946c3ab596ea5f8b3327cb130763d39c95f3e5dd0ca73d21413d5110cbf53 Jul 08 12:06:56.040478 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:56.040434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" event={"ID":"b5971526-bac8-4a3e-9894-fc870ff69081","Type":"ContainerStarted","Data":"593946c3ab596ea5f8b3327cb130763d39c95f3e5dd0ca73d21413d5110cbf53"} Jul 08 12:06:56.041627 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:56.041589 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" event={"ID":"bb21233e-f257-4fd9-9ab7-d53e72d3fda8","Type":"ContainerStarted","Data":"59f380ec7247d1829f4f784890299bbee323e815e1085926936d9422a8574993"} Jul 08 12:06:59.067643 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:59.067602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" event={"ID":"b5971526-bac8-4a3e-9894-fc870ff69081","Type":"ContainerStarted","Data":"f9edc32a1c42a6ac1720e70929f0693fea6d0d8ef05bc27d8094e186630562f1"} Jul 08 12:06:59.070167 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:59.070128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" event={"ID":"bb21233e-f257-4fd9-9ab7-d53e72d3fda8","Type":"ContainerStarted","Data":"e2ad48f8dcaebca594a087c1925403c0d333c0fc7341a260f32d34ea585862eb"} Jul 08 12:06:59.070323 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:59.070219 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:06:59.099429 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:59.099361 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s4shh" podStartSLOduration=0.995745782 podStartE2EDuration="4.099340182s" totalImagesPullingTime="3.1035944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:06:59.098700939 +0000 UTC m=+291.781753738" watchObservedRunningTime="2026-07-08 12:06:59.099340182 +0000 UTC m=+291.782392981" Jul 08 12:06:59.128967 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:06:59.128860 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" podStartSLOduration=1.008958836 podStartE2EDuration="4.128842532s" totalImagesPullingTime="3.119883696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:06:59.126657581 +0000 UTC m=+291.809710379" watchObservedRunningTime="2026-07-08 12:06:59.128842532 +0000 UTC m=+291.811895329" Jul 08 12:07:05.077834 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:05.077800 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h4pds" Jul 08 12:07:07.703653 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:07.703620 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:07:07.707357 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:07.707326 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:07:07.709966 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:07.709936 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 12:07:27.903907 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.903855 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt"] Jul 08 12:07:27.907208 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.907187 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:27.918584 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.918557 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 12:07:27.918749 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.918729 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 12:07:27.919081 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.919065 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 12:07:27.919307 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.919294 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 12:07:27.921826 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.921806 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bp9fb\"" Jul 08 12:07:27.953535 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:27.953494 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt"] Jul 08 12:07:28.030821 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.030784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/998f8cc3-b403-4e61-bad2-86ce337c2a64-webhook-cert\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.030821 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.030822 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/998f8cc3-b403-4e61-bad2-86ce337c2a64-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.031127 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.030974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/998f8cc3-b403-4e61-bad2-86ce337c2a64-manifests\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.031127 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.031069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx85l\" (UniqueName: \"kubernetes.io/projected/998f8cc3-b403-4e61-bad2-86ce337c2a64-kube-api-access-rx85l\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.031127 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.031120 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/998f8cc3-b403-4e61-bad2-86ce337c2a64-tmp\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.131792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.131747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/998f8cc3-b403-4e61-bad2-86ce337c2a64-manifests\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.132019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.131811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx85l\" (UniqueName: \"kubernetes.io/projected/998f8cc3-b403-4e61-bad2-86ce337c2a64-kube-api-access-rx85l\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.132019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.131840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/998f8cc3-b403-4e61-bad2-86ce337c2a64-tmp\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.132019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.131862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/998f8cc3-b403-4e61-bad2-86ce337c2a64-webhook-cert\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.132019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.131880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/998f8cc3-b403-4e61-bad2-86ce337c2a64-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.132225 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.132204 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/998f8cc3-b403-4e61-bad2-86ce337c2a64-manifests\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.134485 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.134450 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/998f8cc3-b403-4e61-bad2-86ce337c2a64-tmp\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.134617 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.134547 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/998f8cc3-b403-4e61-bad2-86ce337c2a64-webhook-cert\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.134658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.134622 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/998f8cc3-b403-4e61-bad2-86ce337c2a64-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.146459 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.146427 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx85l\" (UniqueName: \"kubernetes.io/projected/998f8cc3-b403-4e61-bad2-86ce337c2a64-kube-api-access-rx85l\") pod \"opendatahub-operator-controller-manager-6db55b68b6-j7gtt\" (UID: \"998f8cc3-b403-4e61-bad2-86ce337c2a64\") " pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.218659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.218558 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:28.386053 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.386027 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt"] Jul 08 12:07:28.388361 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:07:28.388330 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998f8cc3_b403_4e61_bad2_86ce337c2a64.slice/crio-85d425a0e6504be9701122b5912a824a5f61df64b3c740a77a4a5d83f741db40 WatchSource:0}: Error finding container 85d425a0e6504be9701122b5912a824a5f61df64b3c740a77a4a5d83f741db40: Status 404 returned error can't find the container with id 85d425a0e6504be9701122b5912a824a5f61df64b3c740a77a4a5d83f741db40 Jul 08 12:07:28.390240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:28.390222 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 12:07:29.202910 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:29.202848 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" event={"ID":"998f8cc3-b403-4e61-bad2-86ce337c2a64","Type":"ContainerStarted","Data":"85d425a0e6504be9701122b5912a824a5f61df64b3c740a77a4a5d83f741db40"} Jul 08 12:07:32.218902 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:32.218848 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" event={"ID":"998f8cc3-b403-4e61-bad2-86ce337c2a64","Type":"ContainerStarted","Data":"7f272efb9b56d5f24fbc102d133399638dec8efd1015021e1e27c3507a70e3c5"} Jul 08 12:07:33.224927 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:33.224867 2560 generic.go:356] "Generic (PLEG): container finished" podID="998f8cc3-b403-4e61-bad2-86ce337c2a64" containerID="7f272efb9b56d5f24fbc102d133399638dec8efd1015021e1e27c3507a70e3c5" exitCode=0 Jul 08 12:07:33.224927 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:33.224931 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" event={"ID":"998f8cc3-b403-4e61-bad2-86ce337c2a64","Type":"ContainerDied","Data":"7f272efb9b56d5f24fbc102d133399638dec8efd1015021e1e27c3507a70e3c5"} Jul 08 12:07:34.231753 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:34.231700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" event={"ID":"998f8cc3-b403-4e61-bad2-86ce337c2a64","Type":"ContainerStarted","Data":"0490a6585162a2dfe947c8900b50f6546666c12f1fc746318f185fc6b837de0c"} Jul 08 12:07:34.232205 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:34.231920 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:34.264256 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:34.264196 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" podStartSLOduration=2.829117013 podStartE2EDuration="7.264179219s" totalImagesPullingTime="3.418466243s" totalInitContainerRuntime="1.016595963s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:07:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:07:34.263338229 +0000 UTC m=+326.946391029" watchObservedRunningTime="2026-07-08 12:07:34.264179219 +0000 UTC m=+326.947232017" Jul 08 12:07:45.239079 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:45.239042 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6db55b68b6-j7gtt" Jul 08 12:07:56.402804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.402759 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9"] Jul 08 12:07:56.411419 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.411386 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.416437 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.416404 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 12:07:56.417367 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.417339 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8jlmf\"" Jul 08 12:07:56.417506 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.417348 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 12:07:56.417506 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.417485 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 12:07:56.425880 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.425852 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9"] Jul 08 12:07:56.486645 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.486604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.486968 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd87l\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-kube-api-access-wd87l\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487052 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487088 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487180 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.487532 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.487318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588252 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588252 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd87l\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-kube-api-access-wd87l\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588854 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588738 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.588950 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.588878 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.589150 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.589119 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.589259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.589136 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.589259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.589201 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.591419 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.591384 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.591534 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.591487 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.600247 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.600212 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.600480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.600457 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd87l\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-kube-api-access-wd87l\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6nms9\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.722528 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.722435 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:07:56.871702 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:56.871664 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9"] Jul 08 12:07:56.875051 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:07:56.875019 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310b2a2c_b4b6_4f3a_923e_f19aa537064b.slice/crio-2982ae82aa99a8a4fe440f24b730b799507a9290a7358baeff7647c0066d1644 WatchSource:0}: Error finding container 2982ae82aa99a8a4fe440f24b730b799507a9290a7358baeff7647c0066d1644: Status 404 returned error can't find the container with id 2982ae82aa99a8a4fe440f24b730b799507a9290a7358baeff7647c0066d1644 Jul 08 12:07:57.336352 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.336300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" event={"ID":"310b2a2c-b4b6-4f3a-923e-f19aa537064b","Type":"ContainerStarted","Data":"2982ae82aa99a8a4fe440f24b730b799507a9290a7358baeff7647c0066d1644"} Jul 08 12:07:57.547802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.547765 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c77454567-qttwj"] Jul 08 12:07:57.551469 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.551441 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.555019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.554988 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 12:07:57.555836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.555809 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 12:07:57.555836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.555834 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 12:07:57.556460 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.556117 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 12:07:57.556460 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.556228 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jfd5q\"" Jul 08 12:07:57.556460 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.556338 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 12:07:57.567215 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.567184 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c77454567-qttwj"] Jul 08 12:07:57.598344 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.598021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0575f97-ea37-419f-be23-e1e5bedfae68-metrics-cert\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.598344 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.598074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0575f97-ea37-419f-be23-e1e5bedfae68-manager-config\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.598344 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.598117 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctsgr\" (UniqueName: \"kubernetes.io/projected/b0575f97-ea37-419f-be23-e1e5bedfae68-kube-api-access-ctsgr\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.598344 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.598217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0575f97-ea37-419f-be23-e1e5bedfae68-cert\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.699015 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.698975 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0575f97-ea37-419f-be23-e1e5bedfae68-metrics-cert\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.699218 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.699023 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0575f97-ea37-419f-be23-e1e5bedfae68-manager-config\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.699218 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.699068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctsgr\" (UniqueName: \"kubernetes.io/projected/b0575f97-ea37-419f-be23-e1e5bedfae68-kube-api-access-ctsgr\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.699218 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.699133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0575f97-ea37-419f-be23-e1e5bedfae68-cert\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.700697 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.700611 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0575f97-ea37-419f-be23-e1e5bedfae68-manager-config\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.703174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.703135 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0575f97-ea37-419f-be23-e1e5bedfae68-cert\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.703679 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.703659 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0575f97-ea37-419f-be23-e1e5bedfae68-metrics-cert\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.712448 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.712386 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctsgr\" (UniqueName: \"kubernetes.io/projected/b0575f97-ea37-419f-be23-e1e5bedfae68-kube-api-access-ctsgr\") pod \"lws-controller-manager-6c77454567-qttwj\" (UID: \"b0575f97-ea37-419f-be23-e1e5bedfae68\") " pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.846644 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.846603 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws"] Jul 08 12:07:57.852514 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.852441 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.856093 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.856063 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 12:07:57.857019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.856815 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f2z9h\"" Jul 08 12:07:57.858153 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.858129 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 12:07:57.864092 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.863838 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 12:07:57.864360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.864340 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:07:57.869480 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.869451 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws"] Jul 08 12:07:57.901491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901451 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c45c173-0da4-45cc-8d5f-d95bb569afd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.901491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.901739 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.901739 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901660 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh4gl\" (UniqueName: \"kubernetes.io/projected/0c45c173-0da4-45cc-8d5f-d95bb569afd0-kube-api-access-jh4gl\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.901739 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.901917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:57.901917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:57.901881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003461 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003461 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c45c173-0da4-45cc-8d5f-d95bb569afd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003541 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh4gl\" (UniqueName: \"kubernetes.io/projected/0c45c173-0da4-45cc-8d5f-d95bb569afd0-kube-api-access-jh4gl\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.003670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.003625 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.004652 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.004583 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.006789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.006756 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.007950 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.007479 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.007950 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.007844 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c45c173-0da4-45cc-8d5f-d95bb569afd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.008169 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.008144 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c45c173-0da4-45cc-8d5f-d95bb569afd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.019271 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.019238 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c45c173-0da4-45cc-8d5f-d95bb569afd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.019507 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.019387 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh4gl\" (UniqueName: \"kubernetes.io/projected/0c45c173-0da4-45cc-8d5f-d95bb569afd0-kube-api-access-jh4gl\") pod \"istiod-openshift-gateway-94bb8fbfd-5s8ws\" (UID: \"0c45c173-0da4-45cc-8d5f-d95bb569afd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.145078 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.144978 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c77454567-qttwj"] Jul 08 12:07:58.148194 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:07:58.148159 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0575f97_ea37_419f_be23_e1e5bedfae68.slice/crio-d2ec708e4bf78eea968d378fcd6bd76e05d48c151e988a3ad53f643ca8db2b23 WatchSource:0}: Error finding container d2ec708e4bf78eea968d378fcd6bd76e05d48c151e988a3ad53f643ca8db2b23: Status 404 returned error can't find the container with id d2ec708e4bf78eea968d378fcd6bd76e05d48c151e988a3ad53f643ca8db2b23 Jul 08 12:07:58.166098 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.166056 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:07:58.344329 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.344279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" event={"ID":"b0575f97-ea37-419f-be23-e1e5bedfae68","Type":"ContainerStarted","Data":"d2ec708e4bf78eea968d378fcd6bd76e05d48c151e988a3ad53f643ca8db2b23"} Jul 08 12:07:58.368990 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:58.368964 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws"] Jul 08 12:07:58.370958 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:07:58.370927 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c45c173_0da4_45cc_8d5f_d95bb569afd0.slice/crio-2d0e905d27719c3d0adda6fae756567ea3d49920cd99d438ffe3b2b31fa54ce0 WatchSource:0}: Error finding container 2d0e905d27719c3d0adda6fae756567ea3d49920cd99d438ffe3b2b31fa54ce0: Status 404 returned error can't find the container with id 2d0e905d27719c3d0adda6fae756567ea3d49920cd99d438ffe3b2b31fa54ce0 Jul 08 12:07:59.353641 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:59.353585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" event={"ID":"0c45c173-0da4-45cc-8d5f-d95bb569afd0","Type":"ContainerStarted","Data":"2d0e905d27719c3d0adda6fae756567ea3d49920cd99d438ffe3b2b31fa54ce0"} Jul 08 12:07:59.667003 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:59.666963 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:07:59.667080 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:59.667041 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:07:59.667080 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:07:59.667069 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:08:00.373202 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:00.372224 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" event={"ID":"310b2a2c-b4b6-4f3a-923e-f19aa537064b","Type":"ContainerStarted","Data":"ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8"} Jul 08 12:08:00.418784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:00.417957 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" podStartSLOduration=1.628601986 podStartE2EDuration="4.417931994s" totalImagesPullingTime="2.789330008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:08:00.416086336 +0000 UTC m=+353.099139135" watchObservedRunningTime="2026-07-08 12:08:00.417931994 +0000 UTC m=+353.100984794" Jul 08 12:08:00.723258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:00.723171 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:08:00.724625 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:00.724594 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body= Jul 08 12:08:00.724770 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:00.724724 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" Jul 08 12:08:01.723089 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:01.723048 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body= Jul 08 12:08:01.723571 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:01.723117 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" Jul 08 12:08:02.723304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:02.723263 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body= Jul 08 12:08:02.723852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:02.723336 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" Jul 08 12:08:02.830938 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:02.830873 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9"] Jul 08 12:08:02.931105 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:02.931062 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:08:02.931191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:02.931172 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:08:03.389901 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.389846 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" event={"ID":"0c45c173-0da4-45cc-8d5f-d95bb569afd0","Type":"ContainerStarted","Data":"059b19cb8cdab3d1d7becc6a881427c962cd0812b59e876e19aa5f38b77516b2"} Jul 08 12:08:03.390117 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.389930 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:08:03.391514 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.391480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" event={"ID":"b0575f97-ea37-419f-be23-e1e5bedfae68","Type":"ContainerStarted","Data":"7cc7d60922ef754b0af26881743ec5022052e78baab061dcb1af45171f158a5c"} Jul 08 12:08:03.391615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.391537 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:08:03.391615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.391532 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" containerID="cri-o://ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8" gracePeriod=30 Jul 08 12:08:03.427184 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.427130 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" podStartSLOduration=1.869913504 podStartE2EDuration="6.427112814s" totalImagesPullingTime="4.55719931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:08:03.42333398 +0000 UTC m=+356.106386777" watchObservedRunningTime="2026-07-08 12:08:03.427112814 +0000 UTC m=+356.110165612" Jul 08 12:08:03.455359 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:03.455295 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" podStartSLOduration=1.680295749 podStartE2EDuration="6.455272494s" totalImagesPullingTime="4.774976745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:08:03.453879755 +0000 UTC m=+356.136932554" watchObservedRunningTime="2026-07-08 12:08:03.455272494 +0000 UTC m=+356.138325293" Jul 08 12:08:04.397791 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:04.397761 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5s8ws" Jul 08 12:08:08.652956 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.652875 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:08:08.709665 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709627 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-token\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.709665 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709663 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-certs\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.709876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709716 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-envoy\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.709876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709799 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-credential-socket\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.709876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709837 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-socket\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.709876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709858 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-podinfo\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.710073 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709936 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istiod-ca-cert\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.710073 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.709996 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd87l\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-kube-api-access-wd87l\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.710073 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710031 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-data\") pod \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\" (UID: \"310b2a2c-b4b6-4f3a-923e-f19aa537064b\") " Jul 08 12:08:08.710219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710087 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:08:08.710219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710128 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:08:08.710219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710144 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:08:08.710484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710327 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-data" (OuterVolumeSpecName: "istio-data") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:08:08.710484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710344 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.710484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710364 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.710484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710380 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.710484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.710377 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:08:08.712328 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.712294 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-kube-api-access-wd87l" (OuterVolumeSpecName: "kube-api-access-wd87l") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "kube-api-access-wd87l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:08:08.712454 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.712329 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:08:08.712454 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.712329 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-token" (OuterVolumeSpecName: "istio-token") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:08:08.712454 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.712390 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "310b2a2c-b4b6-4f3a-923e-f19aa537064b" (UID: "310b2a2c-b4b6-4f3a-923e-f19aa537064b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:08:08.812683 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.812646 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd87l\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-kube-api-access-wd87l\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.812683 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.812679 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.812683 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.812689 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.812935 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.812698 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.812935 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.812706 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:08.812935 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:08.812713 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310b2a2c-b4b6-4f3a-923e-f19aa537064b-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:08:09.419817 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.419776 2560 generic.go:356] "Generic (PLEG): container finished" podID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerID="ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8" exitCode=0 Jul 08 12:08:09.420038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.419823 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" event={"ID":"310b2a2c-b4b6-4f3a-923e-f19aa537064b","Type":"ContainerDied","Data":"ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8"} Jul 08 12:08:09.420038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.419842 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" Jul 08 12:08:09.420038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.419877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9" event={"ID":"310b2a2c-b4b6-4f3a-923e-f19aa537064b","Type":"ContainerDied","Data":"2982ae82aa99a8a4fe440f24b730b799507a9290a7358baeff7647c0066d1644"} Jul 08 12:08:09.420038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.419928 2560 scope.go:117] "RemoveContainer" containerID="ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8" Jul 08 12:08:09.429651 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.429632 2560 scope.go:117] "RemoveContainer" containerID="ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8" Jul 08 12:08:09.430056 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:08:09.430023 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8\": container with ID starting with ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8 not found: ID does not exist" containerID="ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8" Jul 08 12:08:09.430174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.430067 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8"} err="failed to get container status \"ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8\": rpc error: code = NotFound desc = could not find container \"ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8\": container with ID starting with ffd5b389239bd771b24ea31d933f4286c62148236f7afe9676a1c56d8da6d4d8 not found: ID does not exist" Jul 08 12:08:09.453744 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.453709 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9"] Jul 08 12:08:09.458385 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.458356 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6nms9"] Jul 08 12:08:09.889023 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:09.888989 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" path="/var/lib/kubelet/pods/310b2a2c-b4b6-4f3a-923e-f19aa537064b/volumes" Jul 08 12:08:14.398047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:08:14.398010 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c77454567-qttwj" Jul 08 12:09:43.350133 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.350026 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj"] Jul 08 12:09:43.350663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.350402 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" Jul 08 12:09:43.350663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.350418 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" Jul 08 12:09:43.350663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.350502 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="310b2a2c-b4b6-4f3a-923e-f19aa537064b" containerName="istio-proxy" Jul 08 12:09:43.353973 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.353947 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.356958 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.356928 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 12:09:43.357120 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.357011 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 12:09:43.357120 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.357039 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9x4hk\"" Jul 08 12:09:43.373016 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.372982 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj"] Jul 08 12:09:43.430759 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.430720 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2klg\" (UniqueName: \"kubernetes.io/projected/067ac498-676b-4220-a983-38ac98ed7c45-kube-api-access-m2klg\") pod \"kuadrant-operator-controller-manager-86878c769f-6t2mj\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.430999 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.430923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/067ac498-676b-4220-a983-38ac98ed7c45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6t2mj\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.531622 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.531579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2klg\" (UniqueName: \"kubernetes.io/projected/067ac498-676b-4220-a983-38ac98ed7c45-kube-api-access-m2klg\") pod \"kuadrant-operator-controller-manager-86878c769f-6t2mj\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.531622 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.531629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/067ac498-676b-4220-a983-38ac98ed7c45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6t2mj\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.532027 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.532010 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/067ac498-676b-4220-a983-38ac98ed7c45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6t2mj\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.546443 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.546402 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2klg\" (UniqueName: \"kubernetes.io/projected/067ac498-676b-4220-a983-38ac98ed7c45-kube-api-access-m2klg\") pod \"kuadrant-operator-controller-manager-86878c769f-6t2mj\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.666003 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.665873 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:43.837266 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:43.837241 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj"] Jul 08 12:09:43.839644 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:09:43.839614 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067ac498_676b_4220_a983_38ac98ed7c45.slice/crio-51485c5cfa0c1246fcfee03c900eeb470dc046704ac31b8ef24c51a8b3bfc14f WatchSource:0}: Error finding container 51485c5cfa0c1246fcfee03c900eeb470dc046704ac31b8ef24c51a8b3bfc14f: Status 404 returned error can't find the container with id 51485c5cfa0c1246fcfee03c900eeb470dc046704ac31b8ef24c51a8b3bfc14f Jul 08 12:09:44.836217 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:44.836182 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" event={"ID":"067ac498-676b-4220-a983-38ac98ed7c45","Type":"ContainerStarted","Data":"51485c5cfa0c1246fcfee03c900eeb470dc046704ac31b8ef24c51a8b3bfc14f"} Jul 08 12:09:51.869616 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:51.869573 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" event={"ID":"067ac498-676b-4220-a983-38ac98ed7c45","Type":"ContainerStarted","Data":"c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4"} Jul 08 12:09:51.870078 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:51.869728 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:09:51.905721 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:09:51.905663 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" podStartSLOduration=1.3538877550000001 podStartE2EDuration="8.90564689s" totalImagesPullingTime="7.551759135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:09:51.904017031 +0000 UTC m=+464.587069829" watchObservedRunningTime="2026-07-08 12:09:51.90564689 +0000 UTC m=+464.588699689" Jul 08 12:10:02.876876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:02.876838 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:10:19.358902 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.358840 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m"] Jul 08 12:10:19.365439 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.365414 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.370953 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.370923 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8xg2l\"" Jul 08 12:10:19.380804 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.380770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m"] Jul 08 12:10:19.458187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458192 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458221 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458375 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458479 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458412 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458479 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458443 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458545 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458545 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hst8s\" (UniqueName: \"kubernetes.io/projected/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-kube-api-access-hst8s\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.458545 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.458542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.559800 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.559760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.559800 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.559801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.559931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.559983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hst8s\" (UniqueName: \"kubernetes.io/projected/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-kube-api-access-hst8s\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560347 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560347 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560347 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560253 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560503 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560377 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560558 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560505 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560558 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560517 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.560656 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.560636 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.562654 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.562630 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.562854 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.562837 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.569992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.569947 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.569992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.569965 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hst8s\" (UniqueName: \"kubernetes.io/projected/fefdcb1b-5ab4-4a74-8056-9490a2b3eec4-kube-api-access-hst8s\") pod \"maas-default-gateway-openshift-default-56654dcf7-wnz9m\" (UID: \"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.676437 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.676333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:19.828832 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.828806 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m"] Jul 08 12:10:19.831759 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:10:19.831734 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefdcb1b_5ab4_4a74_8056_9490a2b3eec4.slice/crio-0d5a5e93908ffb15df756df8ecf66c470af9716265ea0b2b5c6528ce3668d5e9 WatchSource:0}: Error finding container 0d5a5e93908ffb15df756df8ecf66c470af9716265ea0b2b5c6528ce3668d5e9: Status 404 returned error can't find the container with id 0d5a5e93908ffb15df756df8ecf66c470af9716265ea0b2b5c6528ce3668d5e9 Jul 08 12:10:19.833996 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.833964 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:10:19.834071 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.834036 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:10:19.834071 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.834067 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:10:19.997387 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.997278 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" event={"ID":"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4","Type":"ContainerStarted","Data":"5bfef71cfe9b4ed4da8b11e6d5bcd98e33cee4239e43ee5696c3c3f9d7786af5"} Jul 08 12:10:19.997387 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:19.997360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" event={"ID":"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4","Type":"ContainerStarted","Data":"0d5a5e93908ffb15df756df8ecf66c470af9716265ea0b2b5c6528ce3668d5e9"} Jul 08 12:10:20.029232 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:20.029161 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" podStartSLOduration=1.029138171 podStartE2EDuration="1.029138171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:10:20.026130429 +0000 UTC m=+492.709183228" watchObservedRunningTime="2026-07-08 12:10:20.029138171 +0000 UTC m=+492.712190971" Jul 08 12:10:20.676860 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:20.676816 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:20.682023 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:20.681994 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:21.001921 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:21.001802 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:10:21.002944 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:10:21.002922 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:11:46.400167 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:46.400129 2560 generic.go:356] "Generic (PLEG): container finished" podID="fefdcb1b-5ab4-4a74-8056-9490a2b3eec4" containerID="5bfef71cfe9b4ed4da8b11e6d5bcd98e33cee4239e43ee5696c3c3f9d7786af5" exitCode=0 Jul 08 12:11:46.400697 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:46.400209 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" event={"ID":"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4","Type":"ContainerDied","Data":"5bfef71cfe9b4ed4da8b11e6d5bcd98e33cee4239e43ee5696c3c3f9d7786af5"} Jul 08 12:11:46.400765 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:46.400707 2560 scope.go:117] "RemoveContainer" containerID="5bfef71cfe9b4ed4da8b11e6d5bcd98e33cee4239e43ee5696c3c3f9d7786af5" Jul 08 12:11:46.401734 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:46.401708 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:11:46.401789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:46.401775 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:11:46.401825 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:46.401808 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:11:47.408107 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:47.408073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" event={"ID":"fefdcb1b-5ab4-4a74-8056-9490a2b3eec4","Type":"ContainerStarted","Data":"c8e0e957928a7a9d1e39648537cfc2606622c34a606e982c3b7af30afbe1853e"} Jul 08 12:11:47.676812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:47.676717 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:11:47.677715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:47.677687 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wnz9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 08 12:11:47.677827 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:47.677753 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" podUID="fefdcb1b-5ab4-4a74-8056-9490a2b3eec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 08 12:11:48.677013 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:48.676973 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wnz9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 08 12:11:48.677427 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:48.677031 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" podUID="fefdcb1b-5ab4-4a74-8056-9490a2b3eec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 08 12:11:49.677084 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:49.677038 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wnz9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 08 12:11:49.677508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:49.677103 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" podUID="fefdcb1b-5ab4-4a74-8056-9490a2b3eec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 08 12:11:50.677789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:50.677740 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wnz9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 08 12:11:50.678308 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:50.677809 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" podUID="fefdcb1b-5ab4-4a74-8056-9490a2b3eec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 08 12:11:51.002150 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:51.002053 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:11:51.681155 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:51.681119 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:11:51.682166 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:51.682147 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wnz9m" Jul 08 12:11:56.419405 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.419362 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4"] Jul 08 12:11:56.423210 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.423181 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427169 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427352 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427352 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427231 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f4pdd\"" Jul 08 12:11:56.427352 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427234 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427352 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/210cad19-2921-4dbb-b4ee-03f88b28fc57-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427352 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz56j\" (UniqueName: \"kubernetes.io/projected/210cad19-2921-4dbb-b4ee-03f88b28fc57-kube-api-access-wz56j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427373 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.427671 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427617 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 12:11:56.427671 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427626 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 12:11:56.427671 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.427655 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 12:11:56.437065 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.437035 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4"] Jul 08 12:11:56.528328 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528328 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528593 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528593 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/210cad19-2921-4dbb-b4ee-03f88b28fc57-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528593 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528433 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz56j\" (UniqueName: \"kubernetes.io/projected/210cad19-2921-4dbb-b4ee-03f88b28fc57-kube-api-access-wz56j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528593 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528846 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528751 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528846 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528781 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.528846 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.528830 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.531017 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.530994 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/210cad19-2921-4dbb-b4ee-03f88b28fc57-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.531296 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.531276 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/210cad19-2921-4dbb-b4ee-03f88b28fc57-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.538764 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.538735 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz56j\" (UniqueName: \"kubernetes.io/projected/210cad19-2921-4dbb-b4ee-03f88b28fc57-kube-api-access-wz56j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4\" (UID: \"210cad19-2921-4dbb-b4ee-03f88b28fc57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.735574 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.735476 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:11:56.897275 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:56.897233 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4"] Jul 08 12:11:56.899737 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:11:56.899706 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210cad19_2921_4dbb_b4ee_03f88b28fc57.slice/crio-947c7b0d84f05764eb94b0c746ff301fc4b3fcfb9aeb7a663b71aa7a6ac32c51 WatchSource:0}: Error finding container 947c7b0d84f05764eb94b0c746ff301fc4b3fcfb9aeb7a663b71aa7a6ac32c51: Status 404 returned error can't find the container with id 947c7b0d84f05764eb94b0c746ff301fc4b3fcfb9aeb7a663b71aa7a6ac32c51 Jul 08 12:11:57.456960 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:11:57.456920 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" event={"ID":"210cad19-2921-4dbb-b4ee-03f88b28fc57","Type":"ContainerStarted","Data":"947c7b0d84f05764eb94b0c746ff301fc4b3fcfb9aeb7a663b71aa7a6ac32c51"} Jul 08 12:12:00.474568 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:00.474517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" event={"ID":"210cad19-2921-4dbb-b4ee-03f88b28fc57","Type":"ContainerStarted","Data":"26fcc0d6f369462d2823d6489c4f223236329a8843a20266225c688b246aeb7a"} Jul 08 12:12:00.497350 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:00.497295 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" podStartSLOduration=1.934593564 podStartE2EDuration="4.497277266s" totalImagesPullingTime="2.562683702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:12:00.496062106 +0000 UTC m=+593.179114905" watchObservedRunningTime="2026-07-08 12:12:00.497277266 +0000 UTC m=+593.180330063" Jul 08 12:12:06.736356 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:06.736291 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:12:06.736844 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:06.736388 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:12:06.749060 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:06.749028 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:12:07.519607 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:07.519575 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4" Jul 08 12:12:07.736615 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:07.736584 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:12:07.737063 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:07.736835 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:12:35.821011 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.820918 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5"] Jul 08 12:12:35.824740 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.824713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:35.827953 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.827925 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 12:12:35.835954 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.835921 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5"] Jul 08 12:12:35.975461 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.975419 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:35.975658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.975475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:35.975658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.975511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b420741-1366-452f-829a-03e2da89e47c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:35.975658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.975534 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:35.975658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.975596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49fnk\" (UniqueName: \"kubernetes.io/projected/6b420741-1366-452f-829a-03e2da89e47c-kube-api-access-49fnk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:35.975658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:35.975647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.076731 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.076637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.076731 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.076704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.076977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.076735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.076977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.076760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b420741-1366-452f-829a-03e2da89e47c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.076977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.076782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.077118 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.076973 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49fnk\" (UniqueName: \"kubernetes.io/projected/6b420741-1366-452f-829a-03e2da89e47c-kube-api-access-49fnk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.077210 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.077191 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.077265 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.077231 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.077317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.077282 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.079240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.079215 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b420741-1366-452f-829a-03e2da89e47c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.079483 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.079464 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b420741-1366-452f-829a-03e2da89e47c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.087574 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.087541 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49fnk\" (UniqueName: \"kubernetes.io/projected/6b420741-1366-452f-829a-03e2da89e47c-kube-api-access-49fnk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5\" (UID: \"6b420741-1366-452f-829a-03e2da89e47c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.135626 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.135587 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:36.291056 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.291029 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5"] Jul 08 12:12:36.294023 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:12:36.293986 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b420741_1366_452f_829a_03e2da89e47c.slice/crio-c23cdf95bd8d650db31ec05d3277b211191328fe59ba25e5a32aeae94383ce52 WatchSource:0}: Error finding container c23cdf95bd8d650db31ec05d3277b211191328fe59ba25e5a32aeae94383ce52: Status 404 returned error can't find the container with id c23cdf95bd8d650db31ec05d3277b211191328fe59ba25e5a32aeae94383ce52 Jul 08 12:12:36.295426 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.295407 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 12:12:36.652688 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.652648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" event={"ID":"6b420741-1366-452f-829a-03e2da89e47c","Type":"ContainerStarted","Data":"69b8102bde5356d7c44bb5145e2a9fb9ae1ac8bbc2df330bbc77bf465959ef0e"} Jul 08 12:12:36.652940 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.652708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" event={"ID":"6b420741-1366-452f-829a-03e2da89e47c","Type":"ContainerStarted","Data":"c23cdf95bd8d650db31ec05d3277b211191328fe59ba25e5a32aeae94383ce52"} Jul 08 12:12:36.676132 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:36.676075 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" podStartSLOduration=1.497986332 podStartE2EDuration="1.676053364s" totalImagesPullingTime="178.067032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:12:36.674501604 +0000 UTC m=+629.357554404" watchObservedRunningTime="2026-07-08 12:12:36.676053364 +0000 UTC m=+629.359106162" Jul 08 12:12:46.136225 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:46.136183 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:46.136658 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:46.136254 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:46.149056 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:46.149019 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:12:46.713409 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:12:46.713378 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5" Jul 08 12:15:00.177777 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.177735 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v"] Jul 08 12:15:00.180924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.180900 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:15:00.183975 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.183945 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 12:15:00.183975 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.183966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mp9nf\"" Jul 08 12:15:00.184997 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.184979 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 12:15:00.193571 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.193536 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v"] Jul 08 12:15:00.258497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.258454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bblcv\" (UniqueName: \"kubernetes.io/projected/6114b8e8-02cd-4398-9c21-30ad37169a92-kube-api-access-bblcv\") pod \"maas-api-key-cleanup-29725215-clc4v\" (UID: \"6114b8e8-02cd-4398-9c21-30ad37169a92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:15:00.359296 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.359251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bblcv\" (UniqueName: \"kubernetes.io/projected/6114b8e8-02cd-4398-9c21-30ad37169a92-kube-api-access-bblcv\") pod \"maas-api-key-cleanup-29725215-clc4v\" (UID: \"6114b8e8-02cd-4398-9c21-30ad37169a92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:15:00.369646 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.369612 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bblcv\" (UniqueName: \"kubernetes.io/projected/6114b8e8-02cd-4398-9c21-30ad37169a92-kube-api-access-bblcv\") pod \"maas-api-key-cleanup-29725215-clc4v\" (UID: \"6114b8e8-02cd-4398-9c21-30ad37169a92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:15:00.492434 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.492339 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:15:00.647247 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:00.647213 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v"] Jul 08 12:15:00.649626 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:15:00.649590 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6114b8e8_02cd_4398_9c21_30ad37169a92.slice/crio-eae4759c1b1fad5dca1e93a822aebff6131b080d37558b59bd856f6fb3096952 WatchSource:0}: Error finding container eae4759c1b1fad5dca1e93a822aebff6131b080d37558b59bd856f6fb3096952: Status 404 returned error can't find the container with id eae4759c1b1fad5dca1e93a822aebff6131b080d37558b59bd856f6fb3096952 Jul 08 12:15:01.339277 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:01.339227 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" event={"ID":"6114b8e8-02cd-4398-9c21-30ad37169a92","Type":"ContainerStarted","Data":"eae4759c1b1fad5dca1e93a822aebff6131b080d37558b59bd856f6fb3096952"} Jul 08 12:15:03.352715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:15:03.352670 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" event={"ID":"6114b8e8-02cd-4398-9c21-30ad37169a92","Type":"ContainerStarted","Data":"c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22"} Jul 08 12:16:00.014947 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:00.014863 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" podStartSLOduration=57.694039705 podStartE2EDuration="1m0.014847371s" totalImagesPullingTime="2.320807666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:15:03.389008226 +0000 UTC m=+776.072061027" watchObservedRunningTime="2026-07-08 12:16:00.014847371 +0000 UTC m=+832.697900168" Jul 08 12:16:00.016765 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:00.016733 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v"] Jul 08 12:16:00.017028 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:00.017001 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" podUID="6114b8e8-02cd-4398-9c21-30ad37169a92" containerName="cleanup" containerID="cri-o://c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22" gracePeriod=30 Jul 08 12:16:30.205776 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.205735 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:16:30.291597 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.291493 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bblcv\" (UniqueName: \"kubernetes.io/projected/6114b8e8-02cd-4398-9c21-30ad37169a92-kube-api-access-bblcv\") pod \"6114b8e8-02cd-4398-9c21-30ad37169a92\" (UID: \"6114b8e8-02cd-4398-9c21-30ad37169a92\") " Jul 08 12:16:30.294024 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.293982 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6114b8e8-02cd-4398-9c21-30ad37169a92-kube-api-access-bblcv" (OuterVolumeSpecName: "kube-api-access-bblcv") pod "6114b8e8-02cd-4398-9c21-30ad37169a92" (UID: "6114b8e8-02cd-4398-9c21-30ad37169a92"). InnerVolumeSpecName "kube-api-access-bblcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:16:30.392655 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.392616 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bblcv\" (UniqueName: \"kubernetes.io/projected/6114b8e8-02cd-4398-9c21-30ad37169a92-kube-api-access-bblcv\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:16:30.782703 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.782667 2560 generic.go:356] "Generic (PLEG): container finished" podID="6114b8e8-02cd-4398-9c21-30ad37169a92" containerID="c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22" exitCode=137 Jul 08 12:16:30.782924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.782711 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" event={"ID":"6114b8e8-02cd-4398-9c21-30ad37169a92","Type":"ContainerDied","Data":"c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22"} Jul 08 12:16:30.782924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.782718 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" Jul 08 12:16:30.782924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.782785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v" event={"ID":"6114b8e8-02cd-4398-9c21-30ad37169a92","Type":"ContainerDied","Data":"eae4759c1b1fad5dca1e93a822aebff6131b080d37558b59bd856f6fb3096952"} Jul 08 12:16:30.782924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.782831 2560 scope.go:117] "RemoveContainer" containerID="c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22" Jul 08 12:16:30.791843 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.791823 2560 scope.go:117] "RemoveContainer" containerID="c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22" Jul 08 12:16:30.792171 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:16:30.792152 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22\": container with ID starting with c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22 not found: ID does not exist" containerID="c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22" Jul 08 12:16:30.792251 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.792181 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22"} err="failed to get container status \"c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22\": rpc error: code = NotFound desc = could not find container \"c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22\": container with ID starting with c4eb1ba9b378322c26beeb4bf1fb17afcd470bde73fde16abcb14439835d9a22 not found: ID does not exist" Jul 08 12:16:30.808301 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.808266 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v"] Jul 08 12:16:30.810963 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:30.810931 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725215-clc4v"] Jul 08 12:16:31.887622 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:16:31.887580 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6114b8e8-02cd-4398-9c21-30ad37169a92" path="/var/lib/kubelet/pods/6114b8e8-02cd-4398-9c21-30ad37169a92/volumes" Jul 08 12:17:07.765669 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:17:07.765585 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:17:07.769753 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:17:07.769725 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:20:53.354972 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:53.354923 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj"] Jul 08 12:20:53.355522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:53.355136 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" podUID="067ac498-676b-4220-a983-38ac98ed7c45" containerName="manager" containerID="cri-o://c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4" gracePeriod=10 Jul 08 12:20:56.721463 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.721433 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:20:56.814606 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.814493 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/067ac498-676b-4220-a983-38ac98ed7c45-extensions-socket-volume\") pod \"067ac498-676b-4220-a983-38ac98ed7c45\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " Jul 08 12:20:56.814606 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.814555 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2klg\" (UniqueName: \"kubernetes.io/projected/067ac498-676b-4220-a983-38ac98ed7c45-kube-api-access-m2klg\") pod \"067ac498-676b-4220-a983-38ac98ed7c45\" (UID: \"067ac498-676b-4220-a983-38ac98ed7c45\") " Jul 08 12:20:56.814939 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.814877 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067ac498-676b-4220-a983-38ac98ed7c45-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "067ac498-676b-4220-a983-38ac98ed7c45" (UID: "067ac498-676b-4220-a983-38ac98ed7c45"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:20:56.817000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.816969 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/067ac498-676b-4220-a983-38ac98ed7c45-kube-api-access-m2klg" (OuterVolumeSpecName: "kube-api-access-m2klg") pod "067ac498-676b-4220-a983-38ac98ed7c45" (UID: "067ac498-676b-4220-a983-38ac98ed7c45"). InnerVolumeSpecName "kube-api-access-m2klg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:20:56.916187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.916148 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/067ac498-676b-4220-a983-38ac98ed7c45-extensions-socket-volume\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:20:56.916187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:56.916185 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2klg\" (UniqueName: \"kubernetes.io/projected/067ac498-676b-4220-a983-38ac98ed7c45-kube-api-access-m2klg\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:20:57.056224 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.056189 2560 generic.go:356] "Generic (PLEG): container finished" podID="067ac498-676b-4220-a983-38ac98ed7c45" containerID="c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4" exitCode=0 Jul 08 12:20:57.056389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.056233 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" Jul 08 12:20:57.056389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.056265 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" event={"ID":"067ac498-676b-4220-a983-38ac98ed7c45","Type":"ContainerDied","Data":"c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4"} Jul 08 12:20:57.056389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.056325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj" event={"ID":"067ac498-676b-4220-a983-38ac98ed7c45","Type":"ContainerDied","Data":"51485c5cfa0c1246fcfee03c900eeb470dc046704ac31b8ef24c51a8b3bfc14f"} Jul 08 12:20:57.056389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.056348 2560 scope.go:117] "RemoveContainer" containerID="c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4" Jul 08 12:20:57.066425 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.066404 2560 scope.go:117] "RemoveContainer" containerID="c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4" Jul 08 12:20:57.066778 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:20:57.066758 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4\": container with ID starting with c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4 not found: ID does not exist" containerID="c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4" Jul 08 12:20:57.066824 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.066792 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4"} err="failed to get container status \"c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4\": rpc error: code = NotFound desc = could not find container \"c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4\": container with ID starting with c02863d7aa37e750174e7dc0896521b593b403f3bf64c9532f86e3203f9906b4 not found: ID does not exist" Jul 08 12:20:57.080105 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.080071 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj"] Jul 08 12:20:57.085357 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.085324 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6t2mj"] Jul 08 12:20:57.887834 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:20:57.887787 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="067ac498-676b-4220-a983-38ac98ed7c45" path="/var/lib/kubelet/pods/067ac498-676b-4220-a983-38ac98ed7c45/volumes" Jul 08 12:21:54.584249 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584210 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l"] Jul 08 12:21:54.586725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584563 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="067ac498-676b-4220-a983-38ac98ed7c45" containerName="manager" Jul 08 12:21:54.586725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584574 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="067ac498-676b-4220-a983-38ac98ed7c45" containerName="manager" Jul 08 12:21:54.586725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584583 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6114b8e8-02cd-4398-9c21-30ad37169a92" containerName="cleanup" Jul 08 12:21:54.586725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584589 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6114b8e8-02cd-4398-9c21-30ad37169a92" containerName="cleanup" Jul 08 12:21:54.586725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584643 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6114b8e8-02cd-4398-9c21-30ad37169a92" containerName="cleanup" Jul 08 12:21:54.586725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.584654 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="067ac498-676b-4220-a983-38ac98ed7c45" containerName="manager" Jul 08 12:21:54.587704 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.587686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.590540 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.590514 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 12:21:54.590688 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.590558 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9x4hk\"" Jul 08 12:21:54.590976 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.590961 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 12:21:54.600323 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.600291 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l"] Jul 08 12:21:54.636068 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.636034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef7ff816-124b-4eef-babe-ba880272ba0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2995l\" (UID: \"ef7ff816-124b-4eef-babe-ba880272ba0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.636252 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.636087 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f4pk\" (UniqueName: \"kubernetes.io/projected/ef7ff816-124b-4eef-babe-ba880272ba0e-kube-api-access-5f4pk\") pod \"kuadrant-operator-controller-manager-86878c769f-2995l\" (UID: \"ef7ff816-124b-4eef-babe-ba880272ba0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.736705 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.736657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef7ff816-124b-4eef-babe-ba880272ba0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2995l\" (UID: \"ef7ff816-124b-4eef-babe-ba880272ba0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.736937 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.736733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f4pk\" (UniqueName: \"kubernetes.io/projected/ef7ff816-124b-4eef-babe-ba880272ba0e-kube-api-access-5f4pk\") pod \"kuadrant-operator-controller-manager-86878c769f-2995l\" (UID: \"ef7ff816-124b-4eef-babe-ba880272ba0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.737121 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.737097 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef7ff816-124b-4eef-babe-ba880272ba0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2995l\" (UID: \"ef7ff816-124b-4eef-babe-ba880272ba0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.746639 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.746606 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f4pk\" (UniqueName: \"kubernetes.io/projected/ef7ff816-124b-4eef-babe-ba880272ba0e-kube-api-access-5f4pk\") pod \"kuadrant-operator-controller-manager-86878c769f-2995l\" (UID: \"ef7ff816-124b-4eef-babe-ba880272ba0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:54.898303 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:54.898197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:55.052747 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:55.052705 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l"] Jul 08 12:21:55.058627 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:21:55.058588 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7ff816_124b_4eef_babe_ba880272ba0e.slice/crio-b26ab9cdfce0dc16f72c697208fe63f5409db2311c0fe41407b8b7b3d6dd641b WatchSource:0}: Error finding container b26ab9cdfce0dc16f72c697208fe63f5409db2311c0fe41407b8b7b3d6dd641b: Status 404 returned error can't find the container with id b26ab9cdfce0dc16f72c697208fe63f5409db2311c0fe41407b8b7b3d6dd641b Jul 08 12:21:55.334453 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:55.334414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" event={"ID":"ef7ff816-124b-4eef-babe-ba880272ba0e","Type":"ContainerStarted","Data":"6151af01dd92d0438350f89e09a89c969f5b39aca2e7796dfb297386c534eac9"} Jul 08 12:21:55.334643 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:55.334467 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" event={"ID":"ef7ff816-124b-4eef-babe-ba880272ba0e","Type":"ContainerStarted","Data":"b26ab9cdfce0dc16f72c697208fe63f5409db2311c0fe41407b8b7b3d6dd641b"} Jul 08 12:21:55.334643 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:55.334520 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:21:55.370709 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:21:55.370650 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" podStartSLOduration=1.370634061 podStartE2EDuration="1.370634061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:21:55.369810449 +0000 UTC m=+1188.052863247" watchObservedRunningTime="2026-07-08 12:21:55.370634061 +0000 UTC m=+1188.053686910" Jul 08 12:22:06.342174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:22:06.342136 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2995l" Jul 08 12:22:07.796769 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:22:07.796735 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:22:07.801207 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:22:07.801181 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:27:07.823322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:27:07.823287 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:27:07.829754 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:27:07.829729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:28:39.523014 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.522970 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns"] Jul 08 12:28:39.526594 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.526569 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.534560 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.534531 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3a1658d8-openshift-default-dockercfg-v95qs\"" Jul 08 12:28:39.549266 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549216 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-certs\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549266 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549270 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-envoy\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-token\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549337 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-credential-socket\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549388 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skkz4\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-kube-api-access-skkz4\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549465 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-data\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-podinfo\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549534 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-socket\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.549629 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.549560 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istiod-ca-cert\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.600720 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.600675 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns"] Jul 08 12:28:39.650496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-certs\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650500 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-envoy\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-token\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650564 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-credential-socket\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skkz4\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-kube-api-access-skkz4\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-data\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-podinfo\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-socket\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.650762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istiod-ca-cert\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.651144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.650971 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-certs\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.651593 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.651566 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-credential-socket\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.651784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.651591 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-data\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.651784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.651618 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istiod-ca-cert\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.652082 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.652058 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-socket\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.654025 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.653994 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-envoy\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.654665 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.654645 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-podinfo\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.660774 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.660734 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-token\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.661205 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.661180 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skkz4\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-kube-api-access-skkz4\") pod \"e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:39.838617 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:39.838514 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:40.004587 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.004552 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns"] Jul 08 12:28:40.005457 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:28:40.005427 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1cd442c_c0cd_4aa0_9095_9fda87326fa5.slice/crio-1f171b38e9179c9686624e157850e25183fb200f90e48bc9e01d866052fc0805 WatchSource:0}: Error finding container 1f171b38e9179c9686624e157850e25183fb200f90e48bc9e01d866052fc0805: Status 404 returned error can't find the container with id 1f171b38e9179c9686624e157850e25183fb200f90e48bc9e01d866052fc0805 Jul 08 12:28:40.007796 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.007757 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:28:40.007870 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.007854 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:28:40.007936 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.007922 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:28:40.259546 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.259507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" event={"ID":"d1cd442c-c0cd-4aa0-9095-9fda87326fa5","Type":"ContainerStarted","Data":"f545ebda64e67ee3cc6aaafae7d33ea848aa82dd4ecfc8b0c067f8c2fc9506b1"} Jul 08 12:28:40.259714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.259576 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" event={"ID":"d1cd442c-c0cd-4aa0-9095-9fda87326fa5","Type":"ContainerStarted","Data":"1f171b38e9179c9686624e157850e25183fb200f90e48bc9e01d866052fc0805"} Jul 08 12:28:40.286603 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.286543 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" podStartSLOduration=1.286527124 podStartE2EDuration="1.286527124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:28:40.285510523 +0000 UTC m=+1592.968563322" watchObservedRunningTime="2026-07-08 12:28:40.286527124 +0000 UTC m=+1592.969579921" Jul 08 12:28:40.838992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:40.838949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:41.844813 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:41.844782 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:42.269732 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:42.269702 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:42.270840 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:42.270814 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:46.003569 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:46.003528 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns"] Jul 08 12:28:46.004169 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:46.003745 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" podUID="d1cd442c-c0cd-4aa0-9095-9fda87326fa5" containerName="istio-proxy" containerID="cri-o://f545ebda64e67ee3cc6aaafae7d33ea848aa82dd4ecfc8b0c067f8c2fc9506b1" gracePeriod=30 Jul 08 12:28:52.321238 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.321203 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1cd442c-c0cd-4aa0-9095-9fda87326fa5" containerID="f545ebda64e67ee3cc6aaafae7d33ea848aa82dd4ecfc8b0c067f8c2fc9506b1" exitCode=0 Jul 08 12:28:52.321651 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.321280 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" event={"ID":"d1cd442c-c0cd-4aa0-9095-9fda87326fa5","Type":"ContainerDied","Data":"f545ebda64e67ee3cc6aaafae7d33ea848aa82dd4ecfc8b0c067f8c2fc9506b1"} Jul 08 12:28:52.373623 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.373591 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:52.466332 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466226 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-envoy\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466332 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466266 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-data\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466332 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466287 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-podinfo\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466332 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466330 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istiod-ca-cert\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466672 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466374 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-socket\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466672 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466466 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-token\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466672 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466502 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-certs\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466672 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466528 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skkz4\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-kube-api-access-skkz4\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466672 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466558 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-credential-socket\") pod \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\" (UID: \"d1cd442c-c0cd-4aa0-9095-9fda87326fa5\") " Jul 08 12:28:52.466977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466755 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:28:52.466977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466822 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:28:52.466977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.466872 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.467133 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.467075 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:28:52.467239 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.467199 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:28:52.467316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.467147 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:28:52.468941 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.468856 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:28:52.468941 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.468867 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:28:52.469465 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.469443 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-kube-api-access-skkz4" (OuterVolumeSpecName: "kube-api-access-skkz4") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "kube-api-access-skkz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:28:52.469525 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.469454 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1cd442c-c0cd-4aa0-9095-9fda87326fa5" (UID: "d1cd442c-c0cd-4aa0-9095-9fda87326fa5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:28:52.568092 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568042 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568092 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568086 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568092 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568100 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skkz4\" (UniqueName: \"kubernetes.io/projected/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-kube-api-access-skkz4\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568342 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568115 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568342 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568129 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568342 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568142 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568342 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568154 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.568342 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.568167 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1cd442c-c0cd-4aa0-9095-9fda87326fa5-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:28:52.639664 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.639622 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds"] Jul 08 12:28:52.640106 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.640082 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1cd442c-c0cd-4aa0-9095-9fda87326fa5" containerName="istio-proxy" Jul 08 12:28:52.640106 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.640105 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1cd442c-c0cd-4aa0-9095-9fda87326fa5" containerName="istio-proxy" Jul 08 12:28:52.640465 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.640177 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1cd442c-c0cd-4aa0-9095-9fda87326fa5" containerName="istio-proxy" Jul 08 12:28:52.643647 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.643613 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.648300 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.648270 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-16272b8a-openshift-default-dockercfg-7w8cd\"" Jul 08 12:28:52.663488 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.663453 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds"] Jul 08 12:28:52.769957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.769916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/389969d1-bd0e-464d-bd23-c5543583d283-istio-podinfo\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770164 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.769975 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-credential-socket\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770164 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-istio-token\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770164 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckdm7\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-kube-api-access-ckdm7\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770164 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770082 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-envoy\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770164 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770105 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-certs\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770166 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-socket\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770193 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-data\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.770360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.770241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/389969d1-bd0e-464d-bd23-c5543583d283-istiod-ca-cert\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/389969d1-bd0e-464d-bd23-c5543583d283-istiod-ca-cert\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/389969d1-bd0e-464d-bd23-c5543583d283-istio-podinfo\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-credential-socket\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-istio-token\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckdm7\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-kube-api-access-ckdm7\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-envoy\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-certs\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-socket\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871384 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-data\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871724 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-data\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871824 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871756 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-certs\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871822 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-credential-socket\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.871996 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.871974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-socket\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.872089 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.872070 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/389969d1-bd0e-464d-bd23-c5543583d283-istiod-ca-cert\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.873927 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.873880 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-envoy\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.874207 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.874187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/389969d1-bd0e-464d-bd23-c5543583d283-istio-podinfo\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.881053 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.881023 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-istio-token\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.881188 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.881072 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckdm7\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-kube-api-access-ckdm7\") pod \"e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:52.957694 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:52.957648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:53.113360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.113284 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds"] Jul 08 12:28:53.115813 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:28:53.115778 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389969d1_bd0e_464d_bd23_c5543583d283.slice/crio-8b09858bb06f55b7b6ce340f39dfec360606644ca356538189dd7e11d74e0693 WatchSource:0}: Error finding container 8b09858bb06f55b7b6ce340f39dfec360606644ca356538189dd7e11d74e0693: Status 404 returned error can't find the container with id 8b09858bb06f55b7b6ce340f39dfec360606644ca356538189dd7e11d74e0693 Jul 08 12:28:53.118151 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.118116 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:28:53.118227 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.118191 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:28:53.118262 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.118247 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:28:53.328776 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.328672 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" Jul 08 12:28:53.328776 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.328700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns" event={"ID":"d1cd442c-c0cd-4aa0-9095-9fda87326fa5","Type":"ContainerDied","Data":"1f171b38e9179c9686624e157850e25183fb200f90e48bc9e01d866052fc0805"} Jul 08 12:28:53.328776 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.328772 2560 scope.go:117] "RemoveContainer" containerID="f545ebda64e67ee3cc6aaafae7d33ea848aa82dd4ecfc8b0c067f8c2fc9506b1" Jul 08 12:28:53.330927 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.330809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" event={"ID":"389969d1-bd0e-464d-bd23-c5543583d283","Type":"ContainerStarted","Data":"d8aba410c049f69ac2d705d298c5c52de6839bb09c26ba0a58bdcefb72f98a29"} Jul 08 12:28:53.330927 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.330871 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" event={"ID":"389969d1-bd0e-464d-bd23-c5543583d283","Type":"ContainerStarted","Data":"8b09858bb06f55b7b6ce340f39dfec360606644ca356538189dd7e11d74e0693"} Jul 08 12:28:53.356865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.356804 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" podStartSLOduration=1.356784091 podStartE2EDuration="1.356784091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:28:53.355098354 +0000 UTC m=+1606.038151153" watchObservedRunningTime="2026-07-08 12:28:53.356784091 +0000 UTC m=+1606.039836949" Jul 08 12:28:53.381474 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.381423 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns"] Jul 08 12:28:53.383834 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.383796 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3a1658d8-openshift-default-57ff6b5d59-lntns"] Jul 08 12:28:53.887203 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.887163 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1cd442c-c0cd-4aa0-9095-9fda87326fa5" path="/var/lib/kubelet/pods/d1cd442c-c0cd-4aa0-9095-9fda87326fa5/volumes" Jul 08 12:28:53.958635 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:53.958594 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:54.964164 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:54.964129 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:55.342796 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:55.342757 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:28:55.343912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:28:55.343861 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:29:05.081538 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.081499 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds"] Jul 08 12:29:05.082097 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.081765 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" podUID="389969d1-bd0e-464d-bd23-c5543583d283" containerName="istio-proxy" containerID="cri-o://d8aba410c049f69ac2d705d298c5c52de6839bb09c26ba0a58bdcefb72f98a29" gracePeriod=30 Jul 08 12:29:05.401357 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.401317 2560 generic.go:356] "Generic (PLEG): container finished" podID="389969d1-bd0e-464d-bd23-c5543583d283" containerID="d8aba410c049f69ac2d705d298c5c52de6839bb09c26ba0a58bdcefb72f98a29" exitCode=0 Jul 08 12:29:05.401556 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.401384 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" event={"ID":"389969d1-bd0e-464d-bd23-c5543583d283","Type":"ContainerDied","Data":"d8aba410c049f69ac2d705d298c5c52de6839bb09c26ba0a58bdcefb72f98a29"} Jul 08 12:29:05.541060 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.541027 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:29:05.707781 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.707744 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-istio-token\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708017 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.707804 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/389969d1-bd0e-464d-bd23-c5543583d283-istio-podinfo\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708017 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.707829 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-credential-socket\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708017 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.707874 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-envoy\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708017 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.707928 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-socket\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708017 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/389969d1-bd0e-464d-bd23-c5543583d283-istiod-ca-cert\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708036 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-data\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708075 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckdm7\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-kube-api-access-ckdm7\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708269 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708102 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-certs\") pod \"389969d1-bd0e-464d-bd23-c5543583d283\" (UID: \"389969d1-bd0e-464d-bd23-c5543583d283\") " Jul 08 12:29:05.708413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708234 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:05.708413 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708326 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:05.708521 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708452 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.708521 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708472 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.708810 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.708644 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/389969d1-bd0e-464d-bd23-c5543583d283-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:29:05.709154 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.709120 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:05.709304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.709273 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-data" (OuterVolumeSpecName: "istio-data") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:05.710415 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.710389 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-istio-token" (OuterVolumeSpecName: "istio-token") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:29:05.710527 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.710414 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/389969d1-bd0e-464d-bd23-c5543583d283-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:29:05.710527 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.710417 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:05.710527 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.710449 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-kube-api-access-ckdm7" (OuterVolumeSpecName: "kube-api-access-ckdm7") pod "389969d1-bd0e-464d-bd23-c5543583d283" (UID: "389969d1-bd0e-464d-bd23-c5543583d283"). InnerVolumeSpecName "kube-api-access-ckdm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:29:05.809307 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809257 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckdm7\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-kube-api-access-ckdm7\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.809307 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809300 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.809307 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809313 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/389969d1-bd0e-464d-bd23-c5543583d283-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.809567 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809323 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/389969d1-bd0e-464d-bd23-c5543583d283-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.809567 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809334 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.809567 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809344 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/389969d1-bd0e-464d-bd23-c5543583d283-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:05.809567 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:05.809354 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/389969d1-bd0e-464d-bd23-c5543583d283-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:06.407781 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:06.407745 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" event={"ID":"389969d1-bd0e-464d-bd23-c5543583d283","Type":"ContainerDied","Data":"8b09858bb06f55b7b6ce340f39dfec360606644ca356538189dd7e11d74e0693"} Jul 08 12:29:06.407781 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:06.407765 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds" Jul 08 12:29:06.408290 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:06.407809 2560 scope.go:117] "RemoveContainer" containerID="d8aba410c049f69ac2d705d298c5c52de6839bb09c26ba0a58bdcefb72f98a29" Jul 08 12:29:06.438965 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:06.438903 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds"] Jul 08 12:29:06.439174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:06.438985 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-16272b8a-openshift-default-ffdcdff69-xqgds"] Jul 08 12:29:07.887060 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:07.887027 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="389969d1-bd0e-464d-bd23-c5543583d283" path="/var/lib/kubelet/pods/389969d1-bd0e-464d-bd23-c5543583d283/volumes" Jul 08 12:29:12.501715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.501672 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb"] Jul 08 12:29:12.502249 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.502105 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="389969d1-bd0e-464d-bd23-c5543583d283" containerName="istio-proxy" Jul 08 12:29:12.502249 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.502126 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="389969d1-bd0e-464d-bd23-c5543583d283" containerName="istio-proxy" Jul 08 12:29:12.502249 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.502215 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="389969d1-bd0e-464d-bd23-c5543583d283" containerName="istio-proxy" Jul 08 12:29:12.506990 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.506954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.510006 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.509969 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3cb60f5b-openshift-default-dockercfg-xfl9k\"" Jul 08 12:29:12.518948 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.518904 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb"] Jul 08 12:29:12.674863 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.674816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-podinfo\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.674863 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.674864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-data\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.674899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-token\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.674994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-credential-socket\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.675030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l55kt\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-kube-api-access-l55kt\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.675066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-socket\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.675092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67518115-8110-4edf-bbe1-c3bd111f8c85-istiod-ca-cert\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675283 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.675142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-envoy\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.675283 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.675209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-certs\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776664 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776620 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-envoy\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776679 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-certs\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-podinfo\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-data\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-token\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-credential-socket\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l55kt\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-kube-api-access-l55kt\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-socket\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.776852 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.776842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67518115-8110-4edf-bbe1-c3bd111f8c85-istiod-ca-cert\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.777452 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.777423 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-certs\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.777543 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.777455 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-credential-socket\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.777543 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.777495 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-data\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.777626 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.777561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-socket\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.777796 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.777779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67518115-8110-4edf-bbe1-c3bd111f8c85-istiod-ca-cert\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.780025 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.779992 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-envoy\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.780191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.780170 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-podinfo\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.787908 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.787864 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-token\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.788051 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.787999 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l55kt\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-kube-api-access-l55kt\") pod \"e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.820740 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.820693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:12.982677 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.982638 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb"] Jul 08 12:29:12.986799 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:29:12.986760 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67518115_8110_4edf_bbe1_c3bd111f8c85.slice/crio-8da9bd60454909148602cf7e886ad85a4f3c29b3bb53292634ae7373f422372f WatchSource:0}: Error finding container 8da9bd60454909148602cf7e886ad85a4f3c29b3bb53292634ae7373f422372f: Status 404 returned error can't find the container with id 8da9bd60454909148602cf7e886ad85a4f3c29b3bb53292634ae7373f422372f Jul 08 12:29:12.989391 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.989357 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:29:12.989488 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.989451 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:29:12.989579 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:12.989485 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:29:13.443240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:13.443147 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" event={"ID":"67518115-8110-4edf-bbe1-c3bd111f8c85","Type":"ContainerStarted","Data":"4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb"} Jul 08 12:29:13.443240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:13.443203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" event={"ID":"67518115-8110-4edf-bbe1-c3bd111f8c85","Type":"ContainerStarted","Data":"8da9bd60454909148602cf7e886ad85a4f3c29b3bb53292634ae7373f422372f"} Jul 08 12:29:13.470105 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:13.470041 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" podStartSLOduration=1.470021706 podStartE2EDuration="1.470021706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:29:13.469238106 +0000 UTC m=+1626.152290933" watchObservedRunningTime="2026-07-08 12:29:13.470021706 +0000 UTC m=+1626.153074503" Jul 08 12:29:13.820816 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:13.820769 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:14.827237 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:14.827196 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:15.453957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:15.453922 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:15.455019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:15.455000 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:18.336910 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:18.336855 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb"] Jul 08 12:29:18.476659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:18.476583 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" podUID="67518115-8110-4edf-bbe1-c3bd111f8c85" containerName="istio-proxy" containerID="cri-o://4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb" gracePeriod=30 Jul 08 12:29:23.731586 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.731542 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:23.763540 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763507 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-certs\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763718 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763575 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-podinfo\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763718 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763601 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l55kt\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-kube-api-access-l55kt\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763718 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763625 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-credential-socket\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763718 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763643 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-token\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763718 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763664 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67518115-8110-4edf-bbe1-c3bd111f8c85-istiod-ca-cert\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763738 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-data\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763763 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-socket\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763819 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-envoy\") pod \"67518115-8110-4edf-bbe1-c3bd111f8c85\" (UID: \"67518115-8110-4edf-bbe1-c3bd111f8c85\") " Jul 08 12:29:23.763977 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.763872 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:23.764177 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.764100 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:23.764177 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.764119 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.764419 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.764386 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-data" (OuterVolumeSpecName: "istio-data") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:23.764634 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.764555 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67518115-8110-4edf-bbe1-c3bd111f8c85-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:29:23.764789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.764742 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:23.766350 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.766311 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-token" (OuterVolumeSpecName: "istio-token") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:29:23.766350 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.766327 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:29:23.766569 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.766546 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:29:23.766611 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.766570 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-kube-api-access-l55kt" (OuterVolumeSpecName: "kube-api-access-l55kt") pod "67518115-8110-4edf-bbe1-c3bd111f8c85" (UID: "67518115-8110-4edf-bbe1-c3bd111f8c85"). InnerVolumeSpecName "kube-api-access-l55kt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:29:23.864997 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.864954 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.864997 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.864990 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.864997 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.864999 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.864997 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.865008 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.865259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.865016 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l55kt\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-kube-api-access-l55kt\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.865259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.865025 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67518115-8110-4edf-bbe1-c3bd111f8c85-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.865259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.865034 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67518115-8110-4edf-bbe1-c3bd111f8c85-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:23.865259 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:23.865041 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67518115-8110-4edf-bbe1-c3bd111f8c85-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:29:24.508496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.508459 2560 generic.go:356] "Generic (PLEG): container finished" podID="67518115-8110-4edf-bbe1-c3bd111f8c85" containerID="4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb" exitCode=0 Jul 08 12:29:24.508692 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.508508 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" Jul 08 12:29:24.508692 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.508539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" event={"ID":"67518115-8110-4edf-bbe1-c3bd111f8c85","Type":"ContainerDied","Data":"4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb"} Jul 08 12:29:24.508692 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.508596 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb" event={"ID":"67518115-8110-4edf-bbe1-c3bd111f8c85","Type":"ContainerDied","Data":"8da9bd60454909148602cf7e886ad85a4f3c29b3bb53292634ae7373f422372f"} Jul 08 12:29:24.508692 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.508629 2560 scope.go:117] "RemoveContainer" containerID="4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb" Jul 08 12:29:24.517979 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.517949 2560 scope.go:117] "RemoveContainer" containerID="4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb" Jul 08 12:29:24.518437 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:29:24.518410 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb\": container with ID starting with 4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb not found: ID does not exist" containerID="4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb" Jul 08 12:29:24.518529 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.518451 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb"} err="failed to get container status \"4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb\": rpc error: code = NotFound desc = could not find container \"4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb\": container with ID starting with 4a3d5905bcd78bebe8238ebb0ab1a2d2b4f27fb8450fe76f1db63fab75e2dfbb not found: ID does not exist" Jul 08 12:29:24.531122 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.531086 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb"] Jul 08 12:29:24.536465 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:24.536430 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3cb60f5b-openshift-default-6b4bddd77f-c2mjb"] Jul 08 12:29:25.886922 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:29:25.886868 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67518115-8110-4edf-bbe1-c3bd111f8c85" path="/var/lib/kubelet/pods/67518115-8110-4edf-bbe1-c3bd111f8c85/volumes" Jul 08 12:30:00.160554 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.160511 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw"] Jul 08 12:30:00.161047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.160855 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67518115-8110-4edf-bbe1-c3bd111f8c85" containerName="istio-proxy" Jul 08 12:30:00.161047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.160867 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="67518115-8110-4edf-bbe1-c3bd111f8c85" containerName="istio-proxy" Jul 08 12:30:00.161047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.160942 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="67518115-8110-4edf-bbe1-c3bd111f8c85" containerName="istio-proxy" Jul 08 12:30:00.164132 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.164109 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:30:00.167541 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.167506 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mp9nf\"" Jul 08 12:30:00.167541 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.167528 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 12:30:00.168461 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.168445 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 12:30:00.176305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.174981 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw"] Jul 08 12:30:00.304279 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.304241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7n4p\" (UniqueName: \"kubernetes.io/projected/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf-kube-api-access-w7n4p\") pod \"maas-api-key-cleanup-29725230-d77qw\" (UID: \"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:30:00.405564 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.405509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7n4p\" (UniqueName: \"kubernetes.io/projected/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf-kube-api-access-w7n4p\") pod \"maas-api-key-cleanup-29725230-d77qw\" (UID: \"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:30:00.417216 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.417127 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7n4p\" (UniqueName: \"kubernetes.io/projected/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf-kube-api-access-w7n4p\") pod \"maas-api-key-cleanup-29725230-d77qw\" (UID: \"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:30:00.476732 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.476681 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:30:00.622955 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.622928 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw"] Jul 08 12:30:00.625729 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:30:00.625703 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2ea7ac_5cf5_4e1c_881a_c42a18200bcf.slice/crio-958e72ee1c0c340520ecc8005f911d1565317d62dc472def02533438e91a6655 WatchSource:0}: Error finding container 958e72ee1c0c340520ecc8005f911d1565317d62dc472def02533438e91a6655: Status 404 returned error can't find the container with id 958e72ee1c0c340520ecc8005f911d1565317d62dc472def02533438e91a6655 Jul 08 12:30:00.690000 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:00.689870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" event={"ID":"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf","Type":"ContainerStarted","Data":"958e72ee1c0c340520ecc8005f911d1565317d62dc472def02533438e91a6655"} Jul 08 12:30:01.696145 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:01.696106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" event={"ID":"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf","Type":"ContainerStarted","Data":"bfc30162d4c133df2e32e96f23379265b295fd39ab44523abbdd867566626511"} Jul 08 12:30:01.716763 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:30:01.716707 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" podStartSLOduration=1.716690745 podStartE2EDuration="1.716690745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:30:01.714270854 +0000 UTC m=+1674.397323653" watchObservedRunningTime="2026-07-08 12:30:01.716690745 +0000 UTC m=+1674.399743542" Jul 08 12:31:00.013813 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:00.013775 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw"] Jul 08 12:31:00.014336 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:00.014023 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" podUID="2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" containerName="cleanup" containerID="cri-o://bfc30162d4c133df2e32e96f23379265b295fd39ab44523abbdd867566626511" gracePeriod=30 Jul 08 12:31:06.221340 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.221297 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk"] Jul 08 12:31:06.225512 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.225481 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.228305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.228273 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-49ca3975-openshift-default-dockercfg-rhpsc\"" Jul 08 12:31:06.236042 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.235996 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk"] Jul 08 12:31:06.282773 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.282722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-data\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.282787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istiod-ca-cert\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.282839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-certs\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.282864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n88p\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-kube-api-access-4n88p\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.282996 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-podinfo\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.283059 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-socket\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.283093 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-credential-socket\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.283123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-token\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.283284 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.283160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-envoy\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384344 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-token\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384344 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-envoy\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384630 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-data\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384630 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384434 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istiod-ca-cert\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384630 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-certs\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384781 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n88p\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-kube-api-access-4n88p\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-podinfo\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384913 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-socket\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.384913 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-credential-socket\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.385066 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384915 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-data\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.385066 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.384943 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-certs\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.385229 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.385204 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-credential-socket\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.385360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.385334 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-socket\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.385462 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.385401 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istiod-ca-cert\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.387851 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.387824 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-envoy\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.388046 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.387985 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-podinfo\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.395277 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.395243 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n88p\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-kube-api-access-4n88p\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.395665 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.395640 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-token\") pod \"e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.538963 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.538907 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:06.693389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.693358 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk"] Jul 08 12:31:06.695719 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:31:06.695692 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b5b74b_e7be_454e_b2a8_2d29c93ff3cb.slice/crio-399875a641421d3df72139ddb8e8d2862ff69b31ea4aa04f93b8134aa05bb16e WatchSource:0}: Error finding container 399875a641421d3df72139ddb8e8d2862ff69b31ea4aa04f93b8134aa05bb16e: Status 404 returned error can't find the container with id 399875a641421d3df72139ddb8e8d2862ff69b31ea4aa04f93b8134aa05bb16e Jul 08 12:31:06.698376 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.698334 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:06.698476 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.698430 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:06.698522 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:06.698473 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:07.004575 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:07.004479 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" event={"ID":"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb","Type":"ContainerStarted","Data":"a82effaee0f03cc264f493dbf3e04bd7b8d72ba903df9cb9ef420bb328024669"} Jul 08 12:31:07.004575 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:07.004539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" event={"ID":"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb","Type":"ContainerStarted","Data":"399875a641421d3df72139ddb8e8d2862ff69b31ea4aa04f93b8134aa05bb16e"} Jul 08 12:31:07.034008 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:07.033946 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" podStartSLOduration=1.033865021 podStartE2EDuration="1.033865021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:31:07.032156956 +0000 UTC m=+1739.715209756" watchObservedRunningTime="2026-07-08 12:31:07.033865021 +0000 UTC m=+1739.716917820" Jul 08 12:31:07.539089 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:07.539046 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:07.545219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:07.545191 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:08.009746 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:08.009719 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:08.010979 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:08.010955 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:23.462378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:23.461323 2560 patch_prober.go:28] interesting pod/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:54244->10.134.0.36:15021: read: connection reset by peer" start-of-body= Jul 08 12:31:23.462378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:23.461418 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:54244->10.134.0.36:15021: read: connection reset by peer" Jul 08 12:31:24.095316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:24.095272 2560 generic.go:356] "Generic (PLEG): container finished" podID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerID="a82effaee0f03cc264f493dbf3e04bd7b8d72ba903df9cb9ef420bb328024669" exitCode=0 Jul 08 12:31:24.095507 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:24.095347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" event={"ID":"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb","Type":"ContainerDied","Data":"a82effaee0f03cc264f493dbf3e04bd7b8d72ba903df9cb9ef420bb328024669"} Jul 08 12:31:24.095778 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:24.095758 2560 scope.go:117] "RemoveContainer" containerID="a82effaee0f03cc264f493dbf3e04bd7b8d72ba903df9cb9ef420bb328024669" Jul 08 12:31:24.096837 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:24.096809 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:24.096932 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:24.096916 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:24.096992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:24.096958 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:25.103835 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:25.103795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" event={"ID":"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb","Type":"ContainerStarted","Data":"fd01978d255ebb0457c094355ecc1916d634afac86f5a181a4b6255740ba81bb"} Jul 08 12:31:25.539204 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:25.539159 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:25.539449 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:25.539425 2560 patch_prober.go:28] interesting pod/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 08 12:31:25.539509 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:25.539491 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 08 12:31:26.539346 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:26.539308 2560 patch_prober.go:28] interesting pod/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 08 12:31:26.539758 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:26.539370 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 08 12:31:27.540143 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:27.540104 2560 patch_prober.go:28] interesting pod/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 08 12:31:27.540598 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:27.540172 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 08 12:31:28.326778 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:28.326727 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk"] Jul 08 12:31:28.327067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:28.327037 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" containerID="cri-o://fd01978d255ebb0457c094355ecc1916d634afac86f5a181a4b6255740ba81bb" gracePeriod=30 Jul 08 12:31:30.130787 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:30.130740 2560 generic.go:356] "Generic (PLEG): container finished" podID="2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" containerID="bfc30162d4c133df2e32e96f23379265b295fd39ab44523abbdd867566626511" exitCode=137 Jul 08 12:31:30.131221 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:30.130818 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" event={"ID":"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf","Type":"ContainerDied","Data":"bfc30162d4c133df2e32e96f23379265b295fd39ab44523abbdd867566626511"} Jul 08 12:31:30.204803 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:30.204766 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:31:30.313824 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:30.313732 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7n4p\" (UniqueName: \"kubernetes.io/projected/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf-kube-api-access-w7n4p\") pod \"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf\" (UID: \"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf\") " Jul 08 12:31:30.316080 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:30.316052 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf-kube-api-access-w7n4p" (OuterVolumeSpecName: "kube-api-access-w7n4p") pod "2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" (UID: "2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf"). InnerVolumeSpecName "kube-api-access-w7n4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:31:30.415100 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:30.415046 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7n4p\" (UniqueName: \"kubernetes.io/projected/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf-kube-api-access-w7n4p\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:31.137130 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:31.137094 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" Jul 08 12:31:31.137566 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:31.137109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw" event={"ID":"2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf","Type":"ContainerDied","Data":"958e72ee1c0c340520ecc8005f911d1565317d62dc472def02533438e91a6655"} Jul 08 12:31:31.137566 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:31.137236 2560 scope.go:117] "RemoveContainer" containerID="bfc30162d4c133df2e32e96f23379265b295fd39ab44523abbdd867566626511" Jul 08 12:31:31.164213 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:31.164176 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw"] Jul 08 12:31:31.168496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:31.168459 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725230-d77qw"] Jul 08 12:31:31.886371 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:31.886334 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" path="/var/lib/kubelet/pods/2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf/volumes" Jul 08 12:31:34.160289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.159799 2560 generic.go:356] "Generic (PLEG): container finished" podID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerID="fd01978d255ebb0457c094355ecc1916d634afac86f5a181a4b6255740ba81bb" exitCode=0 Jul 08 12:31:34.160289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.159980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" event={"ID":"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb","Type":"ContainerDied","Data":"fd01978d255ebb0457c094355ecc1916d634afac86f5a181a4b6255740ba81bb"} Jul 08 12:31:34.160289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.160073 2560 scope.go:117] "RemoveContainer" containerID="a82effaee0f03cc264f493dbf3e04bd7b8d72ba903df9cb9ef420bb328024669" Jul 08 12:31:34.195656 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.195618 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:34.351496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351374 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istiod-ca-cert\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.351496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351458 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-data\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.351496 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351489 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-token\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.351812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351519 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-podinfo\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.351812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351560 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-envoy\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.351812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351637 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n88p\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-kube-api-access-4n88p\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.351812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351710 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-socket\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.352083 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351874 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:31:34.352083 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352046 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:31:34.352190 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.351809 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-certs\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.352272 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352139 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-credential-socket\") pod \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\" (UID: \"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb\") " Jul 08 12:31:34.352447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352202 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:31:34.352524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352354 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:31:34.352680 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352663 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.352738 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352686 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.352738 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352699 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.352738 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352712 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.352836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.352783 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:31:34.354073 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.354036 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:31:34.354192 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.354123 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-kube-api-access-4n88p" (OuterVolumeSpecName: "kube-api-access-4n88p") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "kube-api-access-4n88p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:31:34.354416 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.354396 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:31:34.354473 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.354424 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" (UID: "81b5b74b-e7be-454e-b2a8-2d29c93ff3cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:31:34.453411 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.453371 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.453411 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.453407 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.453411 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.453416 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.453670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.453425 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:34.453670 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:34.453434 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n88p\" (UniqueName: \"kubernetes.io/projected/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb-kube-api-access-4n88p\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:31:35.166419 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:35.166388 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" Jul 08 12:31:35.166917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:35.166394 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk" event={"ID":"81b5b74b-e7be-454e-b2a8-2d29c93ff3cb","Type":"ContainerDied","Data":"399875a641421d3df72139ddb8e8d2862ff69b31ea4aa04f93b8134aa05bb16e"} Jul 08 12:31:35.166917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:35.166518 2560 scope.go:117] "RemoveContainer" containerID="fd01978d255ebb0457c094355ecc1916d634afac86f5a181a4b6255740ba81bb" Jul 08 12:31:35.191395 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:35.191352 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk"] Jul 08 12:31:35.197693 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:35.197657 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-49ca3975-openshift-default-58669d7f87-r2xmk"] Jul 08 12:31:35.886695 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:35.886653 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" path="/var/lib/kubelet/pods/81b5b74b-e7be-454e-b2a8-2d29c93ff3cb/volumes" Jul 08 12:31:49.029068 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029021 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq"] Jul 08 12:31:49.029508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029454 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" Jul 08 12:31:49.029508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029470 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" Jul 08 12:31:49.029508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029501 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" containerName="cleanup" Jul 08 12:31:49.029508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029507 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" containerName="cleanup" Jul 08 12:31:49.029645 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029585 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" Jul 08 12:31:49.029645 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029592 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c2ea7ac-5cf5-4e1c-881a-c42a18200bcf" containerName="cleanup" Jul 08 12:31:49.029645 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.029597 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" Jul 08 12:31:49.032736 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.032712 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.035394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.035364 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cbca5e6f-openshift-default-dockercfg-nrcsd\"" Jul 08 12:31:49.045088 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.045048 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq"] Jul 08 12:31:49.083879 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.083837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-socket\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.083879 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.083897 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-data\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.083943 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-credential-socket\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.083987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-certs\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.084016 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-envoy\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.084037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cclbt\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-kube-api-access-cclbt\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.084086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1841c97-5f5d-4d03-93ab-dea3602ea661-istiod-ca-cert\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084144 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.084122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-token\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.084345 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.084160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-podinfo\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185499 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185461 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1841c97-5f5d-4d03-93ab-dea3602ea661-istiod-ca-cert\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185499 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-token\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-podinfo\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-socket\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-data\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-credential-socket\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-certs\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-envoy\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.185802 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.185755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cclbt\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-kube-api-access-cclbt\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.186223 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.186152 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-data\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.186289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.186221 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-credential-socket\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.186289 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.186251 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-certs\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.186457 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.186433 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1841c97-5f5d-4d03-93ab-dea3602ea661-istiod-ca-cert\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.186688 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.186669 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-socket\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.188409 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.188384 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-envoy\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.188520 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.188503 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-podinfo\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.197124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.197085 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-token\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.197302 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.197204 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cclbt\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-kube-api-access-cclbt\") pod \"e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.346372 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.346272 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:49.502673 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.502640 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq"] Jul 08 12:31:49.505878 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:31:49.505670 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1841c97_5f5d_4d03_93ab_dea3602ea661.slice/crio-32e7088566e60ffe1e9e3756ab91d40befc1fe9fb5288800be02aab5b13f3468 WatchSource:0}: Error finding container 32e7088566e60ffe1e9e3756ab91d40befc1fe9fb5288800be02aab5b13f3468: Status 404 returned error can't find the container with id 32e7088566e60ffe1e9e3756ab91d40befc1fe9fb5288800be02aab5b13f3468 Jul 08 12:31:49.507865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.507828 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:49.507964 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.507950 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:49.508012 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:49.507992 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:31:50.243974 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:50.243935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" event={"ID":"a1841c97-5f5d-4d03-93ab-dea3602ea661","Type":"ContainerStarted","Data":"8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877"} Jul 08 12:31:50.244415 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:50.244000 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" event={"ID":"a1841c97-5f5d-4d03-93ab-dea3602ea661","Type":"ContainerStarted","Data":"32e7088566e60ffe1e9e3756ab91d40befc1fe9fb5288800be02aab5b13f3468"} Jul 08 12:31:50.267084 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:50.267012 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podStartSLOduration=1.266985361 podStartE2EDuration="1.266985361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:31:50.265299759 +0000 UTC m=+1782.948352570" watchObservedRunningTime="2026-07-08 12:31:50.266985361 +0000 UTC m=+1782.950038160" Jul 08 12:31:50.346435 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:50.346389 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:50.348263 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:50.348229 2560 patch_prober.go:28] interesting pod/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 12:31:50.348415 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:50.348295 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 12:31:51.351376 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:51.351336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:52.255581 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:52.255531 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:31:52.256768 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:31:52.256746 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:32:06.333066 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:06.333032 2560 generic.go:356] "Generic (PLEG): container finished" podID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerID="8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877" exitCode=0 Jul 08 12:32:06.333612 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:06.333106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" event={"ID":"a1841c97-5f5d-4d03-93ab-dea3602ea661","Type":"ContainerDied","Data":"8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877"} Jul 08 12:32:06.333688 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:06.333627 2560 scope.go:117] "RemoveContainer" containerID="8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877" Jul 08 12:32:06.334720 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:06.334694 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:32:06.334789 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:06.334770 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:32:06.334830 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:06.334799 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:32:07.256088 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.255949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:32:07.340346 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.340309 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" event={"ID":"a1841c97-5f5d-4d03-93ab-dea3602ea661","Type":"ContainerStarted","Data":"825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06"} Jul 08 12:32:07.347052 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.347006 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:32:07.348506 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.348459 2560 patch_prober.go:28] interesting pod/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 12:32:07.348712 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.348577 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 12:32:07.858025 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.857984 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:32:07.876638 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:07.876608 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:32:08.346745 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:08.346700 2560 patch_prober.go:28] interesting pod/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 12:32:08.347321 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:08.346769 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 12:32:09.347252 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:09.347212 2560 patch_prober.go:28] interesting pod/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 12:32:09.347864 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:09.347830 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 12:32:10.346821 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:10.346768 2560 patch_prober.go:28] interesting pod/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 12:32:10.347090 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:10.346849 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 12:32:11.347648 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:11.347602 2560 patch_prober.go:28] interesting pod/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 12:32:11.348100 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:11.347671 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 12:32:12.351543 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:12.351507 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:32:12.351988 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:12.351815 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:32:12.352542 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:32:12.352524 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:33:34.397862 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:34.397777 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq"] Jul 08 12:33:34.398468 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:34.398030 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" containerID="cri-o://825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06" gracePeriod=30 Jul 08 12:33:39.662511 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.662434 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:33:39.733812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.733772 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-socket\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.733824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-data\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.733849 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-podinfo\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.733910 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cclbt\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-kube-api-access-cclbt\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.733937 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-certs\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734207 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734126 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:33:39.734207 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734183 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-credential-socket\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734311 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734247 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1841c97-5f5d-4d03-93ab-dea3602ea661-istiod-ca-cert\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734311 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734260 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:33:39.734311 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734279 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-envoy\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734326 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-token\") pod \"a1841c97-5f5d-4d03-93ab-dea3602ea661\" (UID: \"a1841c97-5f5d-4d03-93ab-dea3602ea661\") " Jul 08 12:33:39.734466 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734444 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:33:39.734874 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734781 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.735009 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734877 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.735009 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.734909 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.735290 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.735042 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:33:39.735290 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.735088 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1841c97-5f5d-4d03-93ab-dea3602ea661-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:33:39.736430 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.736404 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:33:39.736536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.736411 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:33:39.736536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.736485 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-kube-api-access-cclbt" (OuterVolumeSpecName: "kube-api-access-cclbt") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "kube-api-access-cclbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:33:39.736736 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.736714 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1841c97-5f5d-4d03-93ab-dea3602ea661" (UID: "a1841c97-5f5d-4d03-93ab-dea3602ea661"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:33:39.803106 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.803067 2560 generic.go:356] "Generic (PLEG): container finished" podID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerID="825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06" exitCode=0 Jul 08 12:33:39.803287 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.803125 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" Jul 08 12:33:39.803287 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.803158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" event={"ID":"a1841c97-5f5d-4d03-93ab-dea3602ea661","Type":"ContainerDied","Data":"825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06"} Jul 08 12:33:39.803287 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.803242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq" event={"ID":"a1841c97-5f5d-4d03-93ab-dea3602ea661","Type":"ContainerDied","Data":"32e7088566e60ffe1e9e3756ab91d40befc1fe9fb5288800be02aab5b13f3468"} Jul 08 12:33:39.803287 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.803267 2560 scope.go:117] "RemoveContainer" containerID="825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06" Jul 08 12:33:39.813036 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.813014 2560 scope.go:117] "RemoveContainer" containerID="8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877" Jul 08 12:33:39.822012 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.821991 2560 scope.go:117] "RemoveContainer" containerID="825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06" Jul 08 12:33:39.822349 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:33:39.822323 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06\": container with ID starting with 825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06 not found: ID does not exist" containerID="825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06" Jul 08 12:33:39.822397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.822362 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06"} err="failed to get container status \"825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06\": rpc error: code = NotFound desc = could not find container \"825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06\": container with ID starting with 825247225b8bee18629ca613892ce7f3741d804597f20f7b1e0379dfad898d06 not found: ID does not exist" Jul 08 12:33:39.822397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.822383 2560 scope.go:117] "RemoveContainer" containerID="8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877" Jul 08 12:33:39.822641 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:33:39.822625 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877\": container with ID starting with 8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877 not found: ID does not exist" containerID="8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877" Jul 08 12:33:39.822684 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.822647 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877"} err="failed to get container status \"8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877\": rpc error: code = NotFound desc = could not find container \"8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877\": container with ID starting with 8d908501a0a6486bc70066bf77bf8e1bb91e0b7c6c8b9583f990517018c96877 not found: ID does not exist" Jul 08 12:33:39.827388 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.827353 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq"] Jul 08 12:33:39.831792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.831764 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cbca5e6f-openshift-default-5fc9dfdddc-lf4cq"] Jul 08 12:33:39.836114 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.836074 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1841c97-5f5d-4d03-93ab-dea3602ea661-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.836114 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.836107 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.836114 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.836119 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.836304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.836130 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.836304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.836142 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1841c97-5f5d-4d03-93ab-dea3602ea661-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.836304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.836154 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cclbt\" (UniqueName: \"kubernetes.io/projected/a1841c97-5f5d-4d03-93ab-dea3602ea661-kube-api-access-cclbt\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:33:39.888616 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:33:39.888575 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" path="/var/lib/kubelet/pods/a1841c97-5f5d-4d03-93ab-dea3602ea661/volumes" Jul 08 12:34:30.403447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.403409 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6"] Jul 08 12:34:30.404038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.403932 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" Jul 08 12:34:30.404038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.403951 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" Jul 08 12:34:30.404038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.403977 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" Jul 08 12:34:30.404038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.403985 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" Jul 08 12:34:30.404038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.403997 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" Jul 08 12:34:30.404038 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.404007 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b5b74b-e7be-454e-b2a8-2d29c93ff3cb" containerName="istio-proxy" Jul 08 12:34:30.404439 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.404083 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" Jul 08 12:34:30.407326 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.407301 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.411202 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.411176 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-253a40-openshift-default-dockercfg-76kw6\"" Jul 08 12:34:30.421188 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.421155 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6"] Jul 08 12:34:30.497803 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.497754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bsrx\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-kube-api-access-4bsrx\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.497809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-credential-socket\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.497837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-envoy\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.497929 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-token\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.497987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-data\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.498030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-socket\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498067 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.498052 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e771136d-4c1e-446f-8751-b80c4126a4ce-istiod-ca-cert\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498325 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.498096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-podinfo\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.498325 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.498121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-certs\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598628 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-podinfo\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598628 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-certs\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bsrx\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-kube-api-access-4bsrx\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-credential-socket\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-envoy\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-token\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-data\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-socket\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.598957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.598928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e771136d-4c1e-446f-8751-b80c4126a4ce-istiod-ca-cert\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.599378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.599206 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-certs\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.599378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.599228 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-credential-socket\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.599378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.599350 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-data\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.599544 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.599481 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-socket\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.599687 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.599666 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e771136d-4c1e-446f-8751-b80c4126a4ce-istiod-ca-cert\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.601514 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.601490 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-podinfo\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.601604 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.601543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-envoy\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.609076 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.609041 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-token\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.609215 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.609128 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bsrx\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-kube-api-access-4bsrx\") pod \"e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.718283 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.718181 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:30.871951 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.871913 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6"] Jul 08 12:34:30.875753 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:34:30.875721 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode771136d_4c1e_446f_8751_b80c4126a4ce.slice/crio-c47c17cf037c211004dc333cfff88bcf060515922bca80a4ee34c62625dfd2c3 WatchSource:0}: Error finding container c47c17cf037c211004dc333cfff88bcf060515922bca80a4ee34c62625dfd2c3: Status 404 returned error can't find the container with id c47c17cf037c211004dc333cfff88bcf060515922bca80a4ee34c62625dfd2c3 Jul 08 12:34:30.878113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.878075 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:30.878189 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.878154 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:30.878189 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:30.878183 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:31.040447 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:31.040403 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" event={"ID":"e771136d-4c1e-446f-8751-b80c4126a4ce","Type":"ContainerStarted","Data":"4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e"} Jul 08 12:34:31.040646 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:31.040480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" event={"ID":"e771136d-4c1e-446f-8751-b80c4126a4ce","Type":"ContainerStarted","Data":"c47c17cf037c211004dc333cfff88bcf060515922bca80a4ee34c62625dfd2c3"} Jul 08 12:34:31.062928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:31.062846 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podStartSLOduration=1.0628273990000001 podStartE2EDuration="1.062827399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:34:31.061602172 +0000 UTC m=+1943.744654982" watchObservedRunningTime="2026-07-08 12:34:31.062827399 +0000 UTC m=+1943.745880197" Jul 08 12:34:31.719204 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:31.719161 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:31.724397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:31.724365 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:32.045145 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:32.045110 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:32.046091 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:32.046068 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:38.158490 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.158447 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9"] Jul 08 12:34:38.159125 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.159102 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1841c97-5f5d-4d03-93ab-dea3602ea661" containerName="istio-proxy" Jul 08 12:34:38.162543 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.162515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-istio-token\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169494 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169503 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-data\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169757 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169527 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69954cfa-b49d-4002-9d72-503026bf4671-istio-podinfo\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169757 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-certs\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169757 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169663 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-socket\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169757 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169691 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69954cfa-b49d-4002-9d72-503026bf4671-istiod-ca-cert\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.169757 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2drw\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-kube-api-access-g2drw\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.170031 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-credential-socket\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.170031 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169901 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-envoy\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.170031 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.169940 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-85707a-openshift-default-dockercfg-pckph\"" Jul 08 12:34:38.175696 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.175664 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9"] Jul 08 12:34:38.270858 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.270818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-certs\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.270858 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.270866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-socket\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271137 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.270929 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69954cfa-b49d-4002-9d72-503026bf4671-istiod-ca-cert\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271137 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.270972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2drw\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-kube-api-access-g2drw\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271137 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271058 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-credential-socket\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271137 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271090 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-envoy\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271137 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271118 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-istio-token\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271347 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-data\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271347 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69954cfa-b49d-4002-9d72-503026bf4671-istio-podinfo\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271455 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271352 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-socket\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271455 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271376 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-certs\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271675 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271642 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-credential-socket\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.271992 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.271970 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69954cfa-b49d-4002-9d72-503026bf4671-istiod-ca-cert\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.272319 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.272286 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-data\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.274365 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.274339 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69954cfa-b49d-4002-9d72-503026bf4671-istio-podinfo\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.274725 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.274704 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-envoy\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.283731 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.283691 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2drw\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-kube-api-access-g2drw\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.288131 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.288091 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-istio-token\") pod \"e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.476491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.476376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:38.636666 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.636638 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9"] Jul 08 12:34:38.638985 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:34:38.638951 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69954cfa_b49d_4002_9d72_503026bf4671.slice/crio-7ca5be045f89b2f71749baae1825fce06eb595e9e2edc1d197e174cc1dcfe8fa WatchSource:0}: Error finding container 7ca5be045f89b2f71749baae1825fce06eb595e9e2edc1d197e174cc1dcfe8fa: Status 404 returned error can't find the container with id 7ca5be045f89b2f71749baae1825fce06eb595e9e2edc1d197e174cc1dcfe8fa Jul 08 12:34:38.641206 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.641168 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:38.641320 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.641262 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:38.641320 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:38.641308 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:39.085069 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:39.085028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" event={"ID":"69954cfa-b49d-4002-9d72-503026bf4671","Type":"ContainerStarted","Data":"02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa"} Jul 08 12:34:39.085270 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:39.085087 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" event={"ID":"69954cfa-b49d-4002-9d72-503026bf4671","Type":"ContainerStarted","Data":"7ca5be045f89b2f71749baae1825fce06eb595e9e2edc1d197e174cc1dcfe8fa"} Jul 08 12:34:39.107676 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:39.107610 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podStartSLOduration=1.107590634 podStartE2EDuration="1.107590634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:34:39.10586509 +0000 UTC m=+1951.788917888" watchObservedRunningTime="2026-07-08 12:34:39.107590634 +0000 UTC m=+1951.790643431" Jul 08 12:34:39.477185 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:39.477091 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:39.482698 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:39.482666 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:40.089763 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:40.089733 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:40.090714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:40.090691 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:48.087136 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:48.087075 2560 patch_prober.go:28] interesting pod/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 12:34:48.087680 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:48.087161 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 12:34:49.143989 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:49.143941 2560 generic.go:356] "Generic (PLEG): container finished" podID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerID="4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e" exitCode=0 Jul 08 12:34:49.144527 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:49.144017 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" event={"ID":"e771136d-4c1e-446f-8751-b80c4126a4ce","Type":"ContainerDied","Data":"4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e"} Jul 08 12:34:49.144612 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:49.144593 2560 scope.go:117] "RemoveContainer" containerID="4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e" Jul 08 12:34:49.145973 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:49.145939 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:49.146094 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:49.146033 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:49.146094 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:49.146080 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:50.153293 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:50.153234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" event={"ID":"e771136d-4c1e-446f-8751-b80c4126a4ce","Type":"ContainerStarted","Data":"6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef"} Jul 08 12:34:50.719588 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:50.719541 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:50.719784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:50.719658 2560 patch_prober.go:28] interesting pod/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 08 12:34:50.719784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:50.719694 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 08 12:34:51.719580 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:51.719536 2560 patch_prober.go:28] interesting pod/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 08 12:34:51.720027 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:51.719613 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 08 12:34:52.719517 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:52.719482 2560 patch_prober.go:28] interesting pod/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 08 12:34:52.719781 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:52.719753 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 08 12:34:53.719224 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:53.719188 2560 patch_prober.go:28] interesting pod/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 08 12:34:53.719489 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:53.719466 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 08 12:34:54.723373 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:54.723337 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:54.723833 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:54.723636 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:54.724589 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:54.724568 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:34:56.131122 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:56.131070 2560 patch_prober.go:28] interesting pod/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 12:34:56.131559 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:56.131145 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 12:34:58.207318 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:58.207282 2560 generic.go:356] "Generic (PLEG): container finished" podID="69954cfa-b49d-4002-9d72-503026bf4671" containerID="02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa" exitCode=0 Jul 08 12:34:58.207790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:58.207356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" event={"ID":"69954cfa-b49d-4002-9d72-503026bf4671","Type":"ContainerDied","Data":"02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa"} Jul 08 12:34:58.207790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:58.207765 2560 scope.go:117] "RemoveContainer" containerID="02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa" Jul 08 12:34:58.208954 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:58.208922 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:58.209027 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:58.209002 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:58.209064 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:58.209034 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:34:59.214813 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:59.214769 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" event={"ID":"69954cfa-b49d-4002-9d72-503026bf4671","Type":"ContainerStarted","Data":"9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e"} Jul 08 12:34:59.477778 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:59.477665 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:34:59.478013 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:59.477909 2560 patch_prober.go:28] interesting pod/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 12:34:59.478013 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:34:59.477956 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 12:35:00.477266 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:00.477220 2560 patch_prober.go:28] interesting pod/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 12:35:00.477782 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:00.477293 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 12:35:01.477971 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:01.477925 2560 patch_prober.go:28] interesting pod/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 12:35:01.478448 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:01.477990 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 12:35:02.477788 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:02.477741 2560 patch_prober.go:28] interesting pod/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 12:35:02.478031 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:02.477810 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 12:35:03.481536 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:03.481503 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:35:03.482059 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:03.481952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:35:03.482784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:35:03.482763 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:36:59.938367 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:36:59.938327 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6"] Jul 08 12:36:59.938923 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:36:59.938549 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" containerID="cri-o://6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef" gracePeriod=30 Jul 08 12:37:05.202391 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.202312 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:37:05.274004 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.273961 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-credential-socket\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274004 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274012 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-token\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274044 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-data\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274087 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-socket\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274133 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bsrx\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-kube-api-access-4bsrx\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274159 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-podinfo\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274202 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e771136d-4c1e-446f-8751-b80c4126a4ce-istiod-ca-cert\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274237 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-envoy\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274286 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-certs\") pod \"e771136d-4c1e-446f-8751-b80c4126a4ce\" (UID: \"e771136d-4c1e-446f-8751-b80c4126a4ce\") " Jul 08 12:37:05.274524 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274344 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:05.274697 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274569 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.275171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.274881 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:05.275317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.275192 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:05.275317 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.275204 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:05.275479 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.275351 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e771136d-4c1e-446f-8751-b80c4126a4ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:37:05.276760 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.276730 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:05.276964 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.276730 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:05.277159 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.277141 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:37:05.277240 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.277215 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-kube-api-access-4bsrx" (OuterVolumeSpecName: "kube-api-access-4bsrx") pod "e771136d-4c1e-446f-8751-b80c4126a4ce" (UID: "e771136d-4c1e-446f-8751-b80c4126a4ce"). InnerVolumeSpecName "kube-api-access-4bsrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:05.375948 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375902 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e771136d-4c1e-446f-8751-b80c4126a4ce-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.375948 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375942 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.375948 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375952 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.375948 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375962 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.376279 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375970 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.376279 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375980 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e771136d-4c1e-446f-8751-b80c4126a4ce-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.376279 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.375995 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bsrx\" (UniqueName: \"kubernetes.io/projected/e771136d-4c1e-446f-8751-b80c4126a4ce-kube-api-access-4bsrx\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.376279 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.376005 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e771136d-4c1e-446f-8751-b80c4126a4ce-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:05.844419 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.844378 2560 generic.go:356] "Generic (PLEG): container finished" podID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerID="6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef" exitCode=0 Jul 08 12:37:05.844600 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.844436 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" Jul 08 12:37:05.844600 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.844463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" event={"ID":"e771136d-4c1e-446f-8751-b80c4126a4ce","Type":"ContainerDied","Data":"6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef"} Jul 08 12:37:05.844600 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.844525 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6" event={"ID":"e771136d-4c1e-446f-8751-b80c4126a4ce","Type":"ContainerDied","Data":"c47c17cf037c211004dc333cfff88bcf060515922bca80a4ee34c62625dfd2c3"} Jul 08 12:37:05.844600 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.844549 2560 scope.go:117] "RemoveContainer" containerID="6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef" Jul 08 12:37:05.855028 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.855001 2560 scope.go:117] "RemoveContainer" containerID="4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e" Jul 08 12:37:05.865471 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.865440 2560 scope.go:117] "RemoveContainer" containerID="6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef" Jul 08 12:37:05.866129 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:05.866100 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef\": container with ID starting with 6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef not found: ID does not exist" containerID="6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef" Jul 08 12:37:05.866267 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.866141 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef"} err="failed to get container status \"6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef\": rpc error: code = NotFound desc = could not find container \"6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef\": container with ID starting with 6f8b88789e98f64eae62fc2daea24015458627ea8071b00d08578ffa9f9cb4ef not found: ID does not exist" Jul 08 12:37:05.866267 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.866225 2560 scope.go:117] "RemoveContainer" containerID="4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e" Jul 08 12:37:05.866649 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:05.866621 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e\": container with ID starting with 4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e not found: ID does not exist" containerID="4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e" Jul 08 12:37:05.866727 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.866659 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e"} err="failed to get container status \"4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e\": rpc error: code = NotFound desc = could not find container \"4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e\": container with ID starting with 4d299e295b902f62cf138938fb4cb8390eaccc7b97e0e412f2e822fec903530e not found: ID does not exist" Jul 08 12:37:05.868339 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.868318 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6"] Jul 08 12:37:05.872538 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.872504 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-253a40-openshift-default-64c884c695-h8ss6"] Jul 08 12:37:05.887947 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:05.887877 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" path="/var/lib/kubelet/pods/e771136d-4c1e-446f-8751-b80c4126a4ce/volumes" Jul 08 12:37:07.196440 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:07.196396 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9"] Jul 08 12:37:07.196846 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:07.196675 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" containerID="cri-o://9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e" gracePeriod=30 Jul 08 12:37:07.895759 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:07.895724 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:37:07.906062 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:07.906033 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkcwp_03e45ff0-4e75-4b46-bed1-2d447ad8afc1/ovn-acl-logging/0.log" Jul 08 12:37:08.087258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.087212 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf"] Jul 08 12:37:08.087575 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.087562 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" Jul 08 12:37:08.087657 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.087576 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" Jul 08 12:37:08.087657 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.087587 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" Jul 08 12:37:08.087657 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.087593 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" Jul 08 12:37:08.087798 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.087657 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" Jul 08 12:37:08.092425 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.092398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.096253 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.096217 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c1cfd9-openshift-default-dockercfg-dzq44\"" Jul 08 12:37:08.118249 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.118212 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf"] Jul 08 12:37:08.201871 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.201767 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-podinfo\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.201871 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.201815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbc84\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-kube-api-access-cbc84\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.201871 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.201850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe727473-6dc5-417a-9a9e-726e3e37afc1-istiod-ca-cert\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.202501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.201958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-envoy\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.202501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.201987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-token\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.202501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.202075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-certs\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.202501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.202129 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-socket\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.202501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.202159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-credential-socket\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.202501 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.202178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-data\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.302836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.302793 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-socket\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.302836 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.302840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-credential-socket\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303115 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.302860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-data\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303115 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.302970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-podinfo\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303115 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.302996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbc84\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-kube-api-access-cbc84\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303115 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe727473-6dc5-417a-9a9e-726e3e37afc1-istiod-ca-cert\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303115 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-envoy\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303115 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-token\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303404 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-certs\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.303404 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303286 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-socket\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.304007 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303589 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-certs\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.304007 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303592 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-credential-socket\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.304007 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303690 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-data\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.304007 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.303850 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe727473-6dc5-417a-9a9e-726e3e37afc1-istiod-ca-cert\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.305571 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.305546 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-envoy\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.305931 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.305878 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-podinfo\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.315187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.315141 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-token\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.315430 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.315288 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbc84\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-kube-api-access-cbc84\") pod \"e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.404818 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.404773 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:08.562808 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.562776 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf"] Jul 08 12:37:08.565692 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:37:08.565659 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe727473_6dc5_417a_9a9e_726e3e37afc1.slice/crio-1c60f0439e8b710d9593c5fb9aeb6155ef45a0c224d756545c39c20a797f9831 WatchSource:0}: Error finding container 1c60f0439e8b710d9593c5fb9aeb6155ef45a0c224d756545c39c20a797f9831: Status 404 returned error can't find the container with id 1c60f0439e8b710d9593c5fb9aeb6155ef45a0c224d756545c39c20a797f9831 Jul 08 12:37:08.567990 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.567951 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:08.568085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.568029 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:08.568085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.568059 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:08.866618 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.866581 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" event={"ID":"fe727473-6dc5-417a-9a9e-726e3e37afc1","Type":"ContainerStarted","Data":"15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748"} Jul 08 12:37:08.866815 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.866638 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" event={"ID":"fe727473-6dc5-417a-9a9e-726e3e37afc1","Type":"ContainerStarted","Data":"1c60f0439e8b710d9593c5fb9aeb6155ef45a0c224d756545c39c20a797f9831"} Jul 08 12:37:08.891286 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:08.891177 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podStartSLOduration=0.891158161 podStartE2EDuration="891.158161ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:37:08.890373132 +0000 UTC m=+2101.573425926" watchObservedRunningTime="2026-07-08 12:37:08.891158161 +0000 UTC m=+2101.574210958" Jul 08 12:37:09.404959 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:09.404913 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:09.410233 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:09.410199 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:09.871911 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:09.871851 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:09.872799 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:09.872778 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:12.474270 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.474232 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:37:12.544990 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.544944 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69954cfa-b49d-4002-9d72-503026bf4671-istio-podinfo\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545175 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545002 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2drw\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-kube-api-access-g2drw\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545175 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545056 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-data\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545175 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545082 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-socket\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545175 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545104 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69954cfa-b49d-4002-9d72-503026bf4671-istiod-ca-cert\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545429 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545227 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-certs\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545429 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545290 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-envoy\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545429 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545365 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-credential-socket\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545429 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545426 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-istio-token\") pod \"69954cfa-b49d-4002-9d72-503026bf4671\" (UID: \"69954cfa-b49d-4002-9d72-503026bf4671\") " Jul 08 12:37:12.545724 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.545686 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:12.546346 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.546305 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:12.546465 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.546439 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:12.546719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.546618 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-data" (OuterVolumeSpecName: "istio-data") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:12.546719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.546679 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69954cfa-b49d-4002-9d72-503026bf4671-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:37:12.547792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.547754 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69954cfa-b49d-4002-9d72-503026bf4671-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:37:12.547792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.547767 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-kube-api-access-g2drw" (OuterVolumeSpecName: "kube-api-access-g2drw") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "kube-api-access-g2drw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:12.547930 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.547831 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:12.548015 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.547999 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-istio-token" (OuterVolumeSpecName: "istio-token") pod "69954cfa-b49d-4002-9d72-503026bf4671" (UID: "69954cfa-b49d-4002-9d72-503026bf4671"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:12.646606 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646564 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646606 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646604 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646606 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646614 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69954cfa-b49d-4002-9d72-503026bf4671-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646859 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646623 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2drw\" (UniqueName: \"kubernetes.io/projected/69954cfa-b49d-4002-9d72-503026bf4671-kube-api-access-g2drw\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646859 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646635 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646859 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646643 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646859 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646651 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69954cfa-b49d-4002-9d72-503026bf4671-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646859 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646664 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.646859 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.646676 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69954cfa-b49d-4002-9d72-503026bf4671-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:12.887354 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.887320 2560 generic.go:356] "Generic (PLEG): container finished" podID="69954cfa-b49d-4002-9d72-503026bf4671" containerID="9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e" exitCode=0 Jul 08 12:37:12.887582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.887364 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" event={"ID":"69954cfa-b49d-4002-9d72-503026bf4671","Type":"ContainerDied","Data":"9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e"} Jul 08 12:37:12.887582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.887419 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" event={"ID":"69954cfa-b49d-4002-9d72-503026bf4671","Type":"ContainerDied","Data":"7ca5be045f89b2f71749baae1825fce06eb595e9e2edc1d197e174cc1dcfe8fa"} Jul 08 12:37:12.887582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.887426 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9" Jul 08 12:37:12.887582 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.887448 2560 scope.go:117] "RemoveContainer" containerID="9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e" Jul 08 12:37:12.897608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.897585 2560 scope.go:117] "RemoveContainer" containerID="02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa" Jul 08 12:37:12.907348 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.907319 2560 scope.go:117] "RemoveContainer" containerID="9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e" Jul 08 12:37:12.907679 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:12.907661 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e\": container with ID starting with 9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e not found: ID does not exist" containerID="9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e" Jul 08 12:37:12.907728 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.907688 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e"} err="failed to get container status \"9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e\": rpc error: code = NotFound desc = could not find container \"9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e\": container with ID starting with 9dd3bcea8cd3e080e6351cb8855beb19a9775bfe48f5a8341461e062f975a62e not found: ID does not exist" Jul 08 12:37:12.907728 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.907705 2560 scope.go:117] "RemoveContainer" containerID="02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa" Jul 08 12:37:12.908019 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:12.907996 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa\": container with ID starting with 02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa not found: ID does not exist" containerID="02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa" Jul 08 12:37:12.908072 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.908029 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa"} err="failed to get container status \"02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa\": rpc error: code = NotFound desc = could not find container \"02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa\": container with ID starting with 02bc74c9cb053cd6aa26a148ceee8d6935df11c3c4324ccc67036ab41413cbfa not found: ID does not exist" Jul 08 12:37:12.912171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.912128 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9"] Jul 08 12:37:12.918265 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:12.918230 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-85707a-openshift-default-5c77c86dc-s82h9"] Jul 08 12:37:13.887865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:13.887834 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69954cfa-b49d-4002-9d72-503026bf4671" path="/var/lib/kubelet/pods/69954cfa-b49d-4002-9d72-503026bf4671/volumes" Jul 08 12:37:16.131279 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131241 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd"] Jul 08 12:37:16.131710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131626 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" Jul 08 12:37:16.131710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131639 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" Jul 08 12:37:16.131710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131655 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" Jul 08 12:37:16.131710 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131661 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" Jul 08 12:37:16.131865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131726 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" Jul 08 12:37:16.131865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131736 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69954cfa-b49d-4002-9d72-503026bf4671" containerName="istio-proxy" Jul 08 12:37:16.131865 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.131744 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e771136d-4c1e-446f-8751-b80c4126a4ce" containerName="istio-proxy" Jul 08 12:37:16.136540 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.136508 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.140998 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.140964 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f4d1bb-openshift-default-dockercfg-4pj9b\"" Jul 08 12:37:16.155428 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.155392 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd"] Jul 08 12:37:16.181508 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-socket\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istiod-ca-cert\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-certs\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181571 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-data\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzksx\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-kube-api-access-zzksx\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-credential-socket\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-podinfo\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181962 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-token\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.181962 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.181796 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-envoy\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.282957 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.282870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzksx\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-kube-api-access-zzksx\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.282976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-credential-socket\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.282999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-podinfo\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-token\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-envoy\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283102 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-socket\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283139 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istiod-ca-cert\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283174 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283163 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-certs\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283568 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-data\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283568 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283438 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-credential-socket\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283682 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283629 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-certs\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283682 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283643 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-socket\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283682 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-data\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.283965 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.283945 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istiod-ca-cert\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.285641 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.285619 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-envoy\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.285775 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.285755 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-podinfo\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.293290 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.293184 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-token\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.293474 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.293348 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzksx\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-kube-api-access-zzksx\") pod \"e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.449608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.449514 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:16.602092 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.602062 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd"] Jul 08 12:37:16.604744 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:37:16.604702 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc69d3f_fcd9_42db_bf7a_61b3f825778f.slice/crio-97b4adbc9f7d1db7408cca2ac1c0c04b0dc8d0972ec44d875d9708009750a2cc WatchSource:0}: Error finding container 97b4adbc9f7d1db7408cca2ac1c0c04b0dc8d0972ec44d875d9708009750a2cc: Status 404 returned error can't find the container with id 97b4adbc9f7d1db7408cca2ac1c0c04b0dc8d0972ec44d875d9708009750a2cc Jul 08 12:37:16.607542 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.607501 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:16.607663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.607598 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:16.607663 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.607641 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:16.909151 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.909106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" event={"ID":"0fc69d3f-fcd9-42db-bf7a-61b3f825778f","Type":"ContainerStarted","Data":"c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213"} Jul 08 12:37:16.909322 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.909177 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" event={"ID":"0fc69d3f-fcd9-42db-bf7a-61b3f825778f","Type":"ContainerStarted","Data":"97b4adbc9f7d1db7408cca2ac1c0c04b0dc8d0972ec44d875d9708009750a2cc"} Jul 08 12:37:16.934254 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:16.934194 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" podStartSLOduration=0.934173432 podStartE2EDuration="934.173432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:37:16.932554792 +0000 UTC m=+2109.615607591" watchObservedRunningTime="2026-07-08 12:37:16.934173432 +0000 UTC m=+2109.617226230" Jul 08 12:37:17.450360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:17.450320 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:17.456018 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:17.455986 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:17.913550 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:17.913506 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:17.914719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:17.914695 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:25.914112 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:25.914068 2560 patch_prober.go:28] interesting pod/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 12:37:25.914597 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:25.914141 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 12:37:27.969483 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:27.969378 2560 generic.go:356] "Generic (PLEG): container finished" podID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerID="15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748" exitCode=0 Jul 08 12:37:27.969483 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:27.969447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" event={"ID":"fe727473-6dc5-417a-9a9e-726e3e37afc1","Type":"ContainerDied","Data":"15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748"} Jul 08 12:37:27.969953 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:27.969898 2560 scope.go:117] "RemoveContainer" containerID="15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748" Jul 08 12:37:27.970991 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:27.970960 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:27.971103 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:27.971033 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:27.971103 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:27.971074 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:28.977119 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:28.977081 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" event={"ID":"fe727473-6dc5-417a-9a9e-726e3e37afc1","Type":"ContainerStarted","Data":"70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb"} Jul 08 12:37:29.405711 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:29.405665 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:29.405959 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:29.405920 2560 patch_prober.go:28] interesting pod/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 12:37:29.406034 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:29.405996 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 12:37:30.405488 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:30.405439 2560 patch_prober.go:28] interesting pod/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 12:37:30.405945 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:30.405500 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 12:37:31.406073 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:31.406034 2560 patch_prober.go:28] interesting pod/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 12:37:31.406479 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:31.406108 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 12:37:32.405372 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:32.405329 2560 patch_prober.go:28] interesting pod/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 12:37:32.405660 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:32.405616 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 12:37:33.409717 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:33.409683 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:33.410172 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:33.409992 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:33.410841 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:33.410822 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:37.028752 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:37.028719 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerID="c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213" exitCode=0 Jul 08 12:37:37.029292 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:37.028790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" event={"ID":"0fc69d3f-fcd9-42db-bf7a-61b3f825778f","Type":"ContainerDied","Data":"c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213"} Jul 08 12:37:37.029378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:37.029357 2560 scope.go:117] "RemoveContainer" containerID="c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213" Jul 08 12:37:37.030714 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:37.030672 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:37.030812 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:37.030790 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:37.030861 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:37.030845 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:38.035782 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:38.035738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" event={"ID":"0fc69d3f-fcd9-42db-bf7a-61b3f825778f","Type":"ContainerStarted","Data":"20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6"} Jul 08 12:37:38.286819 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:38.286716 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf"] Jul 08 12:37:38.287063 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:38.287010 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" containerID="cri-o://70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb" gracePeriod=30 Jul 08 12:37:38.450534 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:38.450426 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:38.450784 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:38.450750 2560 patch_prober.go:28] interesting pod/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 12:37:38.450857 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:38.450837 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 12:37:39.450497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:39.450450 2560 patch_prober.go:28] interesting pod/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 12:37:39.451030 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:39.450566 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 12:37:40.451088 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:40.451048 2560 patch_prober.go:28] interesting pod/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 12:37:40.451680 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:40.451651 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 12:37:41.452075 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:41.452032 2560 patch_prober.go:28] interesting pod/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 12:37:41.452705 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:41.452665 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 12:37:42.454911 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:42.454854 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:42.455395 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:42.455200 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:42.455924 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:42.455878 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:43.587877 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.587849 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:43.651801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.651765 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-envoy\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.651801 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.651810 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe727473-6dc5-417a-9a9e-726e3e37afc1-istiod-ca-cert\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.651846 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-data\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.651951 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-token\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.651981 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbc84\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-kube-api-access-cbc84\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652004 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-socket\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652036 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-podinfo\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652086 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652058 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-credential-socket\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652407 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-certs\") pod \"fe727473-6dc5-417a-9a9e-726e3e37afc1\" (UID: \"fe727473-6dc5-417a-9a9e-726e3e37afc1\") " Jul 08 12:37:43.652407 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652281 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe727473-6dc5-417a-9a9e-726e3e37afc1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:37:43.652553 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652413 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe727473-6dc5-417a-9a9e-726e3e37afc1-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.652723 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652701 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:43.652876 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652822 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:43.653072 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.652842 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:43.653622 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.653488 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:43.655433 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.655390 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:43.662526 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.662411 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:37:43.662526 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.662414 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-kube-api-access-cbc84" (OuterVolumeSpecName: "kube-api-access-cbc84") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "kube-api-access-cbc84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:43.662526 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.662423 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe727473-6dc5-417a-9a9e-726e3e37afc1" (UID: "fe727473-6dc5-417a-9a9e-726e3e37afc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:43.753258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753211 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753251 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753261 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753272 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753633 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753280 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753633 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753288 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753633 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753298 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbc84\" (UniqueName: \"kubernetes.io/projected/fe727473-6dc5-417a-9a9e-726e3e37afc1-kube-api-access-cbc84\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:43.753633 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:43.753306 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe727473-6dc5-417a-9a9e-726e3e37afc1-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:44.075316 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.075269 2560 generic.go:356] "Generic (PLEG): container finished" podID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerID="70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb" exitCode=0 Jul 08 12:37:44.075469 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.075346 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" Jul 08 12:37:44.075469 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.075351 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" event={"ID":"fe727473-6dc5-417a-9a9e-726e3e37afc1","Type":"ContainerDied","Data":"70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb"} Jul 08 12:37:44.075469 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.075413 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf" event={"ID":"fe727473-6dc5-417a-9a9e-726e3e37afc1","Type":"ContainerDied","Data":"1c60f0439e8b710d9593c5fb9aeb6155ef45a0c224d756545c39c20a797f9831"} Jul 08 12:37:44.075469 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.075436 2560 scope.go:117] "RemoveContainer" containerID="70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb" Jul 08 12:37:44.084689 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.084668 2560 scope.go:117] "RemoveContainer" containerID="15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748" Jul 08 12:37:44.093922 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.093897 2560 scope.go:117] "RemoveContainer" containerID="70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb" Jul 08 12:37:44.094257 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:44.094234 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb\": container with ID starting with 70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb not found: ID does not exist" containerID="70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb" Jul 08 12:37:44.094303 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.094267 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb"} err="failed to get container status \"70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb\": rpc error: code = NotFound desc = could not find container \"70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb\": container with ID starting with 70565368928c5c6af95729429ba6de098d5e5de9555857650416d36f736b65fb not found: ID does not exist" Jul 08 12:37:44.094303 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.094284 2560 scope.go:117] "RemoveContainer" containerID="15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748" Jul 08 12:37:44.094537 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:44.094522 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748\": container with ID starting with 15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748 not found: ID does not exist" containerID="15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748" Jul 08 12:37:44.094637 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.094541 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748"} err="failed to get container status \"15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748\": rpc error: code = NotFound desc = could not find container \"15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748\": container with ID starting with 15bc5bf7a62b84360d6d653d46bdf14dafa8aabc11b60fc7972dec2e7c843748 not found: ID does not exist" Jul 08 12:37:44.102943 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.102856 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf"] Jul 08 12:37:44.106736 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:44.106705 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c1cfd9-openshift-default-57647675-8z9jf"] Jul 08 12:37:45.809628 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:45.809586 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd"] Jul 08 12:37:45.810099 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:45.809839 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" containerID="cri-o://20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6" gracePeriod=30 Jul 08 12:37:45.887764 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:45.887729 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" path="/var/lib/kubelet/pods/fe727473-6dc5-417a-9a9e-726e3e37afc1/volumes" Jul 08 12:37:46.674100 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674053 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr"] Jul 08 12:37:46.674412 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674398 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" Jul 08 12:37:46.674477 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674414 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" Jul 08 12:37:46.674477 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674432 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" Jul 08 12:37:46.674477 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674438 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" Jul 08 12:37:46.674626 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674507 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" Jul 08 12:37:46.674626 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.674515 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe727473-6dc5-417a-9a9e-726e3e37afc1" containerName="istio-proxy" Jul 08 12:37:46.679335 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.679306 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.682730 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.682703 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b4806b-openshift-default-dockercfg-82gkb\"" Jul 08 12:37:46.690792 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.690761 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr"] Jul 08 12:37:46.781177 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-envoy\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-certs\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781279 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bdde77-40a6-47d1-8297-45026f2bb4a3-istiod-ca-cert\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-podinfo\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781397 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781354 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-token\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781616 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-socket\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781616 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-credential-socket\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781616 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-data\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.781776 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.781619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcpcz\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-kube-api-access-mcpcz\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-envoy\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-certs\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bdde77-40a6-47d1-8297-45026f2bb4a3-istiod-ca-cert\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-podinfo\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-token\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-socket\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-credential-socket\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-data\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcpcz\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-kube-api-access-mcpcz\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.882771 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882758 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-socket\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.883187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882783 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-credential-socket\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.883187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.882821 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-data\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.883187 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.883055 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-certs\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.883409 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.883384 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bdde77-40a6-47d1-8297-45026f2bb4a3-istiod-ca-cert\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.884828 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.884804 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-envoy\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.885470 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.885445 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-podinfo\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.893655 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.893624 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-token\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.893850 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.893821 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcpcz\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-kube-api-access-mcpcz\") pod \"e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:46.994049 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:46.993934 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:47.164940 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:47.164910 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr"] Jul 08 12:37:47.167609 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:37:47.167577 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bdde77_40a6_47d1_8297_45026f2bb4a3.slice/crio-9d95fbc4b7f5ae35dac1df45de5e287bb8328bf1e4dab812d541c0ec7ae98909 WatchSource:0}: Error finding container 9d95fbc4b7f5ae35dac1df45de5e287bb8328bf1e4dab812d541c0ec7ae98909: Status 404 returned error can't find the container with id 9d95fbc4b7f5ae35dac1df45de5e287bb8328bf1e4dab812d541c0ec7ae98909 Jul 08 12:37:47.169760 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:47.169726 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:47.169828 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:47.169800 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:47.169923 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:47.169906 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:48.101486 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:48.101444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" event={"ID":"99bdde77-40a6-47d1-8297-45026f2bb4a3","Type":"ContainerStarted","Data":"78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e"} Jul 08 12:37:48.101917 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:48.101503 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" event={"ID":"99bdde77-40a6-47d1-8297-45026f2bb4a3","Type":"ContainerStarted","Data":"9d95fbc4b7f5ae35dac1df45de5e287bb8328bf1e4dab812d541c0ec7ae98909"} Jul 08 12:37:48.125871 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:48.125815 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podStartSLOduration=2.125797644 podStartE2EDuration="2.125797644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:37:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:37:48.124354117 +0000 UTC m=+2140.807406914" watchObservedRunningTime="2026-07-08 12:37:48.125797644 +0000 UTC m=+2140.808850442" Jul 08 12:37:48.994333 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:48.994290 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:49.000248 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:49.000215 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:49.105971 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:49.105937 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:49.107020 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:49.106992 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:37:51.071378 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.071298 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:51.116840 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.116796 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerID="20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6" exitCode=0 Jul 08 12:37:51.117028 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.116923 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" Jul 08 12:37:51.117028 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.116923 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" event={"ID":"0fc69d3f-fcd9-42db-bf7a-61b3f825778f","Type":"ContainerDied","Data":"20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6"} Jul 08 12:37:51.117122 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.117043 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd" event={"ID":"0fc69d3f-fcd9-42db-bf7a-61b3f825778f","Type":"ContainerDied","Data":"97b4adbc9f7d1db7408cca2ac1c0c04b0dc8d0972ec44d875d9708009750a2cc"} Jul 08 12:37:51.117122 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.117068 2560 scope.go:117] "RemoveContainer" containerID="20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6" Jul 08 12:37:51.120491 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120460 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-data\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.120662 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120545 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-credential-socket\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.120662 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120576 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-token\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.120662 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120603 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istiod-ca-cert\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.120662 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120643 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-certs\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.120912 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120790 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:51.120976 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120924 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-podinfo\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.120976 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120931 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:51.121085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.120985 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzksx\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-kube-api-access-zzksx\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.121085 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121039 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-envoy\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.121191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121134 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-socket\") pod \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\" (UID: \"0fc69d3f-fcd9-42db-bf7a-61b3f825778f\") " Jul 08 12:37:51.121191 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121142 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:37:51.121492 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121406 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:51.121492 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121420 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.121492 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121453 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.121492 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121469 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.122048 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.121637 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:51.123857 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.123820 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:37:51.124114 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.124087 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:51.124295 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.124255 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:37:51.124464 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.124332 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-kube-api-access-zzksx" (OuterVolumeSpecName: "kube-api-access-zzksx") pod "0fc69d3f-fcd9-42db-bf7a-61b3f825778f" (UID: "0fc69d3f-fcd9-42db-bf7a-61b3f825778f"). InnerVolumeSpecName "kube-api-access-zzksx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:37:51.127716 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.127692 2560 scope.go:117] "RemoveContainer" containerID="c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213" Jul 08 12:37:51.142573 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.142544 2560 scope.go:117] "RemoveContainer" containerID="20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6" Jul 08 12:37:51.142974 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:51.142952 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6\": container with ID starting with 20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6 not found: ID does not exist" containerID="20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6" Jul 08 12:37:51.143063 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.142985 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6"} err="failed to get container status \"20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6\": rpc error: code = NotFound desc = could not find container \"20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6\": container with ID starting with 20d1fa7ccdd8f55ea8d8fbe19396821c8d133bc0d0871010cd5037dbb6e413c6 not found: ID does not exist" Jul 08 12:37:51.143063 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.143006 2560 scope.go:117] "RemoveContainer" containerID="c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213" Jul 08 12:37:51.143299 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:37:51.143279 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213\": container with ID starting with c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213 not found: ID does not exist" containerID="c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213" Jul 08 12:37:51.143360 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.143304 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213"} err="failed to get container status \"c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213\": rpc error: code = NotFound desc = could not find container \"c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213\": container with ID starting with c36a8e83c717901b8afb42512a75f562b754a1c61deb1c6b9ef5b32137995213 not found: ID does not exist" Jul 08 12:37:51.222143 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.222099 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.222143 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.222141 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzksx\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-kube-api-access-zzksx\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.222389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.222155 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.222389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.222308 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.222389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.222339 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.222389 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.222353 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fc69d3f-fcd9-42db-bf7a-61b3f825778f-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:37:51.443497 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.443457 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd"] Jul 08 12:37:51.449376 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.449340 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f4d1bb-openshift-default-64db98fd59-d9lfd"] Jul 08 12:37:51.886568 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:51.886526 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" path="/var/lib/kubelet/pods/0fc69d3f-fcd9-42db-bf7a-61b3f825778f/volumes" Jul 08 12:37:55.168790 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.168746 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv"] Jul 08 12:37:55.169334 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.169319 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" Jul 08 12:37:55.169422 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.169341 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" Jul 08 12:37:55.169422 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.169357 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" Jul 08 12:37:55.169422 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.169366 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" Jul 08 12:37:55.169576 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.169491 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" Jul 08 12:37:55.173946 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.173918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.176783 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.176750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-58b9d8-openshift-default-dockercfg-kd67l\"" Jul 08 12:37:55.186700 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.186655 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv"] Jul 08 12:37:55.260209 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260149 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw6kq\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-kube-api-access-mw6kq\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260390 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-data\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260390 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260293 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-podinfo\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260390 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-token\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260557 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-socket\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260557 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260469 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-certs\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260557 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-envoy\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260557 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260520 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-credential-socket\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.260740 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.260606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istiod-ca-cert\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362121 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-credential-socket\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istiod-ca-cert\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw6kq\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-kube-api-access-mw6kq\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362197 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-data\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362341 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-podinfo\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362561 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-token\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362561 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362488 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-socket\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362561 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362524 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-certs\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-envoy\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362567 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-credential-socket\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362719 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362616 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-data\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362932 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362820 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-socket\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.362932 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362853 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-certs\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.363047 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.362926 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istiod-ca-cert\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.365107 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.365082 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-envoy\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.365301 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.365282 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-podinfo\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.371990 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.371963 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-token\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.372107 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.372027 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw6kq\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-kube-api-access-mw6kq\") pod \"e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.488471 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.488367 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:55.658388 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.658359 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv"] Jul 08 12:37:55.659690 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:37:55.659657 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655da94c_5622_41cd_bdf7_d4ed4cb112bd.slice/crio-be22c4a4ebed24d37c7f57b7a8ffa9c3ea126f38b25afa589d180a403da481b6 WatchSource:0}: Error finding container be22c4a4ebed24d37c7f57b7a8ffa9c3ea126f38b25afa589d180a403da481b6: Status 404 returned error can't find the container with id be22c4a4ebed24d37c7f57b7a8ffa9c3ea126f38b25afa589d180a403da481b6 Jul 08 12:37:55.662088 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.662051 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:55.662221 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.662162 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:55.662288 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:55.662215 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:37:56.143617 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:56.143576 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" event={"ID":"655da94c-5622-41cd-bdf7-d4ed4cb112bd","Type":"ContainerStarted","Data":"eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813"} Jul 08 12:37:56.143877 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:56.143648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" event={"ID":"655da94c-5622-41cd-bdf7-d4ed4cb112bd","Type":"ContainerStarted","Data":"be22c4a4ebed24d37c7f57b7a8ffa9c3ea126f38b25afa589d180a403da481b6"} Jul 08 12:37:56.167123 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:56.167066 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podStartSLOduration=1.167043324 podStartE2EDuration="1.167043324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 12:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:37:56.165770118 +0000 UTC m=+2148.848822917" watchObservedRunningTime="2026-07-08 12:37:56.167043324 +0000 UTC m=+2148.850096122" Jul 08 12:37:56.489073 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:56.488967 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:56.494523 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:56.494493 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:57.149614 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:57.149584 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:37:57.150844 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:37:57.150818 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:38:04.843875 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:04.843017 2560 patch_prober.go:28] interesting pod/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:51656->10.134.0.42:15021: read: connection reset by peer" start-of-body= Jul 08 12:38:04.843875 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:04.843221 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:51656->10.134.0.42:15021: read: connection reset by peer" Jul 08 12:38:05.204414 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:05.204306 2560 generic.go:356] "Generic (PLEG): container finished" podID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerID="78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e" exitCode=0 Jul 08 12:38:05.204414 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:05.204390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" event={"ID":"99bdde77-40a6-47d1-8297-45026f2bb4a3","Type":"ContainerDied","Data":"78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e"} Jul 08 12:38:05.204958 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:05.204936 2560 scope.go:117] "RemoveContainer" containerID="78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e" Jul 08 12:38:05.206063 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:05.206032 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:38:05.206178 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:05.206106 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:38:05.206178 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:05.206141 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:38:06.212072 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:06.212038 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" event={"ID":"99bdde77-40a6-47d1-8297-45026f2bb4a3","Type":"ContainerStarted","Data":"09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5"} Jul 08 12:38:06.995108 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:06.995059 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:38:06.995437 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:06.995405 2560 patch_prober.go:28] interesting pod/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 12:38:06.995649 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:06.995624 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 12:38:07.995220 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:07.995063 2560 patch_prober.go:28] interesting pod/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 12:38:07.995220 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:07.995137 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 12:38:08.995436 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:08.995383 2560 patch_prober.go:28] interesting pod/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 12:38:08.995995 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:08.995488 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 12:38:09.995010 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:09.994957 2560 patch_prober.go:28] interesting pod/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 12:38:09.995355 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:09.995304 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 12:38:10.998228 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:10.998189 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:38:10.998614 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:10.998497 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:38:10.999383 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:10.999360 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:38:13.192127 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:13.192075 2560 patch_prober.go:28] interesting pod/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 12:38:13.192556 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:13.192154 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 12:38:14.254366 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:14.254324 2560 generic.go:356] "Generic (PLEG): container finished" podID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerID="eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813" exitCode=0 Jul 08 12:38:14.254366 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:14.254368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" event={"ID":"655da94c-5622-41cd-bdf7-d4ed4cb112bd","Type":"ContainerDied","Data":"eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813"} Jul 08 12:38:14.254955 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:14.254868 2560 scope.go:117] "RemoveContainer" containerID="eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813" Jul 08 12:38:14.256064 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:14.256032 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:38:14.256138 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:14.256109 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:38:14.256176 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:14.256148 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 12:38:15.281255 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:15.281212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" event={"ID":"655da94c-5622-41cd-bdf7-d4ed4cb112bd","Type":"ContainerStarted","Data":"f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053"} Jul 08 12:38:15.489478 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:15.489426 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:38:15.489700 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:15.489669 2560 patch_prober.go:28] interesting pod/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 08 12:38:15.489820 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:15.489720 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 08 12:38:16.489460 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:16.489417 2560 patch_prober.go:28] interesting pod/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 08 12:38:16.489980 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:16.489481 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 08 12:38:17.489363 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:17.489319 2560 patch_prober.go:28] interesting pod/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 08 12:38:17.489557 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:17.489385 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 08 12:38:18.489193 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:18.489155 2560 patch_prober.go:28] interesting pod/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 08 12:38:18.489394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:18.489218 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 08 12:38:19.492778 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:19.492743 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:38:19.493198 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:19.493147 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:38:19.493940 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:38:19.493918 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:41:14.757717 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:14.757628 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr"] Jul 08 12:41:14.758222 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:14.757911 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" containerID="cri-o://09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5" gracePeriod=30 Jul 08 12:41:20.025392 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.025186 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:41:20.173795 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.173754 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bdde77-40a6-47d1-8297-45026f2bb4a3-istiod-ca-cert\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.173795 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.173803 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-data\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.173824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-podinfo\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.173841 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-token\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.173917 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-certs\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.173952 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-credential-socket\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.174014 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-envoy\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174113 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.174099 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-socket\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174430 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.174127 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcpcz\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-kube-api-access-mcpcz\") pod \"99bdde77-40a6-47d1-8297-45026f2bb4a3\" (UID: \"99bdde77-40a6-47d1-8297-45026f2bb4a3\") " Jul 08 12:41:20.174430 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.174263 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99bdde77-40a6-47d1-8297-45026f2bb4a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:41:20.174689 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.174657 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:20.175163 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.175093 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:20.175163 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.175122 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:20.175345 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.175245 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bdde77-40a6-47d1-8297-45026f2bb4a3-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.175408 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.175343 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:20.176517 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.176482 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:41:20.176517 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.176499 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:20.176659 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.176513 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:41:20.176700 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.176687 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-kube-api-access-mcpcz" (OuterVolumeSpecName: "kube-api-access-mcpcz") pod "99bdde77-40a6-47d1-8297-45026f2bb4a3" (UID: "99bdde77-40a6-47d1-8297-45026f2bb4a3"). InnerVolumeSpecName "kube-api-access-mcpcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:41:20.184700 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.184661 2560 generic.go:356] "Generic (PLEG): container finished" podID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerID="09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5" exitCode=0 Jul 08 12:41:20.184838 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.184709 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" Jul 08 12:41:20.184838 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.184745 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" event={"ID":"99bdde77-40a6-47d1-8297-45026f2bb4a3","Type":"ContainerDied","Data":"09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5"} Jul 08 12:41:20.184838 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.184813 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr" event={"ID":"99bdde77-40a6-47d1-8297-45026f2bb4a3","Type":"ContainerDied","Data":"9d95fbc4b7f5ae35dac1df45de5e287bb8328bf1e4dab812d541c0ec7ae98909"} Jul 08 12:41:20.185029 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.184845 2560 scope.go:117] "RemoveContainer" containerID="09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5" Jul 08 12:41:20.196293 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.196270 2560 scope.go:117] "RemoveContainer" containerID="78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e" Jul 08 12:41:20.205258 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.205228 2560 scope.go:117] "RemoveContainer" containerID="09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5" Jul 08 12:41:20.205679 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:41:20.205651 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5\": container with ID starting with 09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5 not found: ID does not exist" containerID="09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5" Jul 08 12:41:20.205729 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.205695 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5"} err="failed to get container status \"09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5\": rpc error: code = NotFound desc = could not find container \"09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5\": container with ID starting with 09baf46ef0943db5c8863951a237ceb7d0a9140ada50d1c72336635ddd978fe5 not found: ID does not exist" Jul 08 12:41:20.205729 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.205724 2560 scope.go:117] "RemoveContainer" containerID="78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e" Jul 08 12:41:20.206383 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:41:20.206281 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e\": container with ID starting with 78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e not found: ID does not exist" containerID="78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e" Jul 08 12:41:20.206383 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.206342 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e"} err="failed to get container status \"78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e\": rpc error: code = NotFound desc = could not find container \"78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e\": container with ID starting with 78534c61eba3184204addad15548f5dd205ae39812f38f510ec9862fd282a34e not found: ID does not exist" Jul 08 12:41:20.209254 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.209218 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr"] Jul 08 12:41:20.212932 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.212877 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b4806b-openshift-default-66f594b65f-hcwlr"] Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.275936 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.275975 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcpcz\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-kube-api-access-mcpcz\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.275986 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.275996 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.276004 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.276012 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.276021 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:20.276039 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:20.276028 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bdde77-40a6-47d1-8297-45026f2bb4a3-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:21.889996 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:21.889962 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" path="/var/lib/kubelet/pods/99bdde77-40a6-47d1-8297-45026f2bb4a3/volumes" Jul 08 12:41:21.903262 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:21.903221 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv"] Jul 08 12:41:21.903473 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:21.903452 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" containerID="cri-o://f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053" gracePeriod=30 Jul 08 12:41:27.169699 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.169617 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:41:27.223087 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.223047 2560 generic.go:356] "Generic (PLEG): container finished" podID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerID="f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053" exitCode=0 Jul 08 12:41:27.223304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.223111 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" Jul 08 12:41:27.223304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.223139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" event={"ID":"655da94c-5622-41cd-bdf7-d4ed4cb112bd","Type":"ContainerDied","Data":"f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053"} Jul 08 12:41:27.223304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.223214 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv" event={"ID":"655da94c-5622-41cd-bdf7-d4ed4cb112bd","Type":"ContainerDied","Data":"be22c4a4ebed24d37c7f57b7a8ffa9c3ea126f38b25afa589d180a403da481b6"} Jul 08 12:41:27.223304 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.223248 2560 scope.go:117] "RemoveContainer" containerID="f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053" Jul 08 12:41:27.232579 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.232556 2560 scope.go:117] "RemoveContainer" containerID="eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813" Jul 08 12:41:27.241276 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.241251 2560 scope.go:117] "RemoveContainer" containerID="f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053" Jul 08 12:41:27.241638 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:41:27.241614 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053\": container with ID starting with f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053 not found: ID does not exist" containerID="f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053" Jul 08 12:41:27.241715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.241648 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053"} err="failed to get container status \"f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053\": rpc error: code = NotFound desc = could not find container \"f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053\": container with ID starting with f57a3e8b0624ea16f947915a684b5c5f4fcc986ae01e523212bcd302f7864053 not found: ID does not exist" Jul 08 12:41:27.241715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.241669 2560 scope.go:117] "RemoveContainer" containerID="eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813" Jul 08 12:41:27.241997 ip-10-0-138-160 kubenswrapper[2560]: E0708 12:41:27.241974 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813\": container with ID starting with eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813 not found: ID does not exist" containerID="eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813" Jul 08 12:41:27.242094 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.242003 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813"} err="failed to get container status \"eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813\": rpc error: code = NotFound desc = could not find container \"eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813\": container with ID starting with eb89e570b94eed9b53bc3e4ddb3f5a1230a4052de6e6a2901381f2ac0902e813 not found: ID does not exist" Jul 08 12:41:27.343838 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.343791 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istiod-ca-cert\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.343853 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-token\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.343913 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-credential-socket\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.343934 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-socket\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.343975 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-certs\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344036 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw6kq\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-kube-api-access-mw6kq\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344074 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344062 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-podinfo\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344112 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-data\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344131 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-envoy\") pod \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\" (UID: \"655da94c-5622-41cd-bdf7-d4ed4cb112bd\") " Jul 08 12:41:27.344394 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344321 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 12:41:27.344551 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344424 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:27.344551 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344540 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istiod-ca-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.344674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344560 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-certs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.344674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344616 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:27.344910 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.344867 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:27.345590 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.345558 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:27.346706 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.346673 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 12:41:27.346706 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.346692 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-kube-api-access-mw6kq" (OuterVolumeSpecName: "kube-api-access-mw6kq") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "kube-api-access-mw6kq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:41:27.346908 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.346865 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 12:41:27.346960 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.346873 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "655da94c-5622-41cd-bdf7-d4ed4cb112bd" (UID: "655da94c-5622-41cd-bdf7-d4ed4cb112bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445487 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-workload-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445526 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw6kq\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-kube-api-access-mw6kq\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445540 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-podinfo\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445550 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445560 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-envoy\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445568 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/655da94c-5622-41cd-bdf7-d4ed4cb112bd-istio-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.445585 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.445577 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/655da94c-5622-41cd-bdf7-d4ed4cb112bd-credential-socket\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\"" Jul 08 12:41:27.547638 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.547581 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv"] Jul 08 12:41:27.555487 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.555452 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-58b9d8-openshift-default-7456d4f99-kp9tv"] Jul 08 12:41:27.886529 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:27.886492 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" path="/var/lib/kubelet/pods/655da94c-5622-41cd-bdf7-d4ed4cb112bd/volumes" Jul 08 12:41:28.132150 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:28.132116 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6db55b68b6-j7gtt_998f8cc3-b403-4e61-bad2-86ce337c2a64/copy-manifests/0.log" Jul 08 12:41:28.139359 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:28.139296 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6db55b68b6-j7gtt_998f8cc3-b403-4e61-bad2-86ce337c2a64/manager/0.log" Jul 08 12:41:30.152435 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:30.152402 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2995l_ef7ff816-124b-4eef-babe-ba880272ba0e/manager/0.log" Jul 08 12:41:30.851646 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:30.851598 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5s8ws_0c45c173-0da4-45cc-8d5f-d95bb569afd0/discovery/0.log" Jul 08 12:41:31.201599 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:31.201515 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/1.log" Jul 08 12:41:31.902500 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:31.902461 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9d6t4_210cad19-2921-4dbb-b4ee-03f88b28fc57/main/0.log" Jul 08 12:41:32.253241 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:32.253151 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmfnb5_6b420741-1366-452f-829a-03e2da89e47c/main/0.log" Jul 08 12:41:39.995918 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:39.995865 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rc5xg_ced800b5-7cf8-42d6-b6dc-c8cd5898c813/global-pull-secret-syncer/0.log" Jul 08 12:41:40.122441 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:40.122408 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ndns4_eb3c4f96-1c10-47f9-b03c-76bebeef9fa4/konnectivity-agent/0.log" Jul 08 12:41:40.263451 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:40.263417 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-160.ec2.internal_bbb64130c7b615dfb98b4e6af62c0947/haproxy/0.log" Jul 08 12:41:45.733124 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:45.733087 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2995l_ef7ff816-124b-4eef-babe-ba880272ba0e/manager/0.log" Jul 08 12:41:47.940798 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:47.940764 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-qkbkg_2095e034-764b-4d9c-b86d-0a45ac4d9fad/monitoring-plugin/0.log" Jul 08 12:41:48.060438 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.060406 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f97b_d7229dcd-fb1e-4675-8957-f7f91d94aaf0/node-exporter/0.log" Jul 08 12:41:48.093032 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.092955 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f97b_d7229dcd-fb1e-4675-8957-f7f91d94aaf0/kube-rbac-proxy/0.log" Jul 08 12:41:48.120553 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.120527 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f97b_d7229dcd-fb1e-4675-8957-f7f91d94aaf0/init-textfile/0.log" Jul 08 12:41:48.342743 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.342713 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/prometheus/0.log" Jul 08 12:41:48.382512 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.382434 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/config-reloader/0.log" Jul 08 12:41:48.443184 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.443149 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/thanos-sidecar/0.log" Jul 08 12:41:48.490255 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.490225 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/kube-rbac-proxy-web/0.log" Jul 08 12:41:48.532121 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.532094 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/kube-rbac-proxy/0.log" Jul 08 12:41:48.558690 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.558654 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/kube-rbac-proxy-thanos/0.log" Jul 08 12:41:48.587359 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.587326 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b95f60a5-7097-4596-bb75-304d94d78272/init-config-reloader/0.log" Jul 08 12:41:48.618751 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.618719 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-npnfs_ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae/prometheus-operator/0.log" Jul 08 12:41:48.641998 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:48.641918 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-npnfs_ff4e2f91-4ba1-40ed-8a25-e5e510dcb0ae/kube-rbac-proxy/0.log" Jul 08 12:41:50.307049 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307002 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm"] Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307374 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307387 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307406 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307412 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307423 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307428 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307478 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" Jul 08 12:41:50.307484 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307486 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" Jul 08 12:41:50.307765 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307494 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fc69d3f-fcd9-42db-bf7a-61b3f825778f" containerName="istio-proxy" Jul 08 12:41:50.307765 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307500 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bdde77-40a6-47d1-8297-45026f2bb4a3" containerName="istio-proxy" Jul 08 12:41:50.307765 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.307506 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="655da94c-5622-41cd-bdf7-d4ed4cb112bd" containerName="istio-proxy" Jul 08 12:41:50.310674 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.310643 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.313503 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.313473 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-drpcz\"/\"kube-root-ca.crt\"" Jul 08 12:41:50.313650 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.313504 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-drpcz\"/\"openshift-service-ca.crt\"" Jul 08 12:41:50.314535 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.314519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-drpcz\"/\"default-dockercfg-8xw2t\"" Jul 08 12:41:50.321019 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.320993 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm"] Jul 08 12:41:50.352005 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.351965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-proc\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.352005 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.352018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6zsb\" (UniqueName: \"kubernetes.io/projected/6e514110-a69c-43c5-b87f-bd00a3d46394-kube-api-access-f6zsb\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.352239 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.352069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-lib-modules\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.352239 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.352100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-sys\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.352239 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.352193 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-podres\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453305 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-sys\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-podres\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-proc\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453397 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-sys\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6zsb\" (UniqueName: \"kubernetes.io/projected/6e514110-a69c-43c5-b87f-bd00a3d46394-kube-api-access-f6zsb\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453462 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-proc\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-lib-modules\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453493 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453485 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-podres\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.453723 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.453554 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e514110-a69c-43c5-b87f-bd00a3d46394-lib-modules\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.462558 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.462517 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6zsb\" (UniqueName: \"kubernetes.io/projected/6e514110-a69c-43c5-b87f-bd00a3d46394-kube-api-access-f6zsb\") pod \"perf-node-gather-daemonset-bgqsm\" (UID: \"6e514110-a69c-43c5-b87f-bd00a3d46394\") " pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.621692 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.621591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:50.766416 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:50.766380 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm"] Jul 08 12:41:50.770151 ip-10-0-138-160 kubenswrapper[2560]: W0708 12:41:50.770118 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6e514110_a69c_43c5_b87f_bd00a3d46394.slice/crio-49df3687e65149b659103146bf7d3434046b6b993c3b43265e967da52f07a33a WatchSource:0}: Error finding container 49df3687e65149b659103146bf7d3434046b6b993c3b43265e967da52f07a33a: Status 404 returned error can't find the container with id 49df3687e65149b659103146bf7d3434046b6b993c3b43265e967da52f07a33a Jul 08 12:41:51.215871 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:51.215834 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-xcmhr_95373b22-9f40-43ab-8f9a-e55c26d3abc2/download-server/0.log" Jul 08 12:41:51.342180 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:51.342141 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" event={"ID":"6e514110-a69c-43c5-b87f-bd00a3d46394","Type":"ContainerStarted","Data":"01a58b862d9b475b2eb3b01dfc57f5f77ddbfea51dc94edf77f6dff065c6724f"} Jul 08 12:41:51.342608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:51.342199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" event={"ID":"6e514110-a69c-43c5-b87f-bd00a3d46394","Type":"ContainerStarted","Data":"49df3687e65149b659103146bf7d3434046b6b993c3b43265e967da52f07a33a"} Jul 08 12:41:51.342608 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:51.342229 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:51.362474 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:51.362412 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" podStartSLOduration=1.362395797 podStartE2EDuration="1.362395797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 12:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 12:41:51.360508688 +0000 UTC m=+2384.043561487" watchObservedRunningTime="2026-07-08 12:41:51.362395797 +0000 UTC m=+2384.045448597" Jul 08 12:41:51.812219 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:51.812185 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-f2jpq_fd03dce9-1cb3-4a89-b65e-4828273956ee/volume-data-source-validator/0.log" Jul 08 12:41:52.850336 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:52.850304 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4x886_d593b87b-d888-4119-a222-83aebe7c1b73/dns/0.log" Jul 08 12:41:52.874539 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:52.874507 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4x886_d593b87b-d888-4119-a222-83aebe7c1b73/kube-rbac-proxy/0.log" Jul 08 12:41:53.064806 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:53.064770 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xgt62_453e570f-3d3f-4bdf-8475-dcf1b0bda30e/dns-node-resolver/0.log" Jul 08 12:41:53.639288 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:53.639259 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dmgs9_0f1897f6-14e3-4e50-b3a6-6c15c91ae66c/node-ca/0.log" Jul 08 12:41:54.810773 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:54.810733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5s8ws_0c45c173-0da4-45cc-8d5f-d95bb569afd0/discovery/0.log" Jul 08 12:41:54.893224 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:54.893198 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/0.log" Jul 08 12:41:54.924762 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:54.924720 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/1.log" Jul 08 12:41:55.803294 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:55.803261 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wg898_e24f66af-36b0-4c3e-8baf-c6306b40575b/serve-healthcheck-canary/0.log" Jul 08 12:41:56.324591 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:56.324556 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4sn6j_eb192faa-37da-4e22-90dc-613baf99d463/kube-rbac-proxy/0.log" Jul 08 12:41:56.352171 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:56.352142 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4sn6j_eb192faa-37da-4e22-90dc-613baf99d463/exporter/0.log" Jul 08 12:41:56.378107 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:56.378078 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4sn6j_eb192faa-37da-4e22-90dc-613baf99d463/extractor/0.log" Jul 08 12:41:57.359021 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:57.358984 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-drpcz/perf-node-gather-daemonset-bgqsm" Jul 08 12:41:59.178715 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:59.178624 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6db55b68b6-j7gtt_998f8cc3-b403-4e61-bad2-86ce337c2a64/manager/0.log" Jul 08 12:41:59.203102 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:41:59.203068 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6db55b68b6-j7gtt_998f8cc3-b403-4e61-bad2-86ce337c2a64/copy-manifests/0.log" Jul 08 12:42:00.697233 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:00.697194 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5s8ws_0c45c173-0da4-45cc-8d5f-d95bb569afd0/discovery/0.log" Jul 08 12:42:00.711354 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:00.711318 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c77454567-qttwj_b0575f97-ea37-419f-be23-e1e5bedfae68/manager/0.log" Jul 08 12:42:00.748928 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:00.748821 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/0.log" Jul 08 12:42:00.778551 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:00.778507 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/1.log" Jul 08 12:42:02.170131 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:02.170097 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5s8ws_0c45c173-0da4-45cc-8d5f-d95bb569afd0/discovery/0.log" Jul 08 12:42:02.228819 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:02.228787 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/0.log" Jul 08 12:42:02.258942 ip-10-0-138-160 kubenswrapper[2560]: I0708 12:42:02.258909 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wnz9m_fefdcb1b-5ab4-4a74-8056-9490a2b3eec4/istio-proxy/1.log"