Jul 13 21:27:29.253648 ip-10-0-133-19 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 21:27:29.253661 ip-10-0-133-19 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 21:27:29.253671 ip-10-0-133-19 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 21:27:29.253978 ip-10-0-133-19 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 21:27:39.448841 ip-10-0-133-19 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 21:27:39.448857 ip-10-0-133-19 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cdfd590186964e388f41d4063870c8e3 -- Jul 13 21:29:37.928312 ip-10-0-133-19 systemd[1]: Starting Kubernetes Kubelet... Jul 13 21:29:38.304299 ip-10-0-133-19 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:29:38.304299 ip-10-0-133-19 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 21:29:38.304299 ip-10-0-133-19 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:29:38.304299 ip-10-0-133-19 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 21:29:38.304299 ip-10-0-133-19 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:29:38.304299 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.302737 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307243 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307256 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307261 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307265 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307269 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307273 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307276 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307279 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307282 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307285 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307288 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307291 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307294 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307297 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307300 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307302 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307305 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307307 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:29:38.308485 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307311 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307314 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307317 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307320 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307324 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307327 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307329 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307332 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307340 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307343 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307345 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307348 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307351 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307354 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307357 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307360 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307363 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307365 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307368 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:29:38.309272 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307370 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307373 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307375 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307378 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307380 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307383 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307390 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307393 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307396 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307398 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307400 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307403 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307405 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307408 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307411 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307414 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307416 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307419 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307421 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307423 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:29:38.310043 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307426 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307428 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307431 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307433 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307436 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307438 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307442 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307444 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307447 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307449 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307451 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307454 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307456 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307459 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307461 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307463 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307466 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307468 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307476 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307479 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:29:38.310840 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307481 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307483 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307486 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307488 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307491 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307494 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307496 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307499 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307502 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307504 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307507 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307509 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307512 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307515 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307517 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307520 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307522 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307526 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307529 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307531 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:29:38.311622 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307534 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.307536 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.308972 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.308981 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309000 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309003 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309006 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309009 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309013 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309017 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309020 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309024 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309027 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309029 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309032 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309035 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309038 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309041 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309043 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:29:38.312405 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309046 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309048 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309051 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309053 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309056 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309058 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309061 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309063 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309065 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309068 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309071 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309074 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309077 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309079 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309082 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309084 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309087 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309089 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309091 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:29:38.313146 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309095 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309098 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309101 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309103 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309106 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309108 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309111 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309114 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309116 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309118 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309121 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309123 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309126 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309130 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309132 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309134 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309137 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309140 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309143 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309145 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:29:38.313929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309147 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309150 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309152 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309155 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309157 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309160 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309162 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309176 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309179 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309181 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309184 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309187 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309191 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309194 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309196 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309199 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309219 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309221 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309224 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:29:38.314739 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309227 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309230 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309232 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309235 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309237 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309240 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309244 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309246 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309249 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309251 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309254 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309257 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309259 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309265 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309268 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309270 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309273 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309276 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309278 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309281 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:29:38.315490 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309284 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309287 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309289 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.309292 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310146 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310153 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310156 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310159 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310162 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310165 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310167 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310170 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310173 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310176 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310178 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310181 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310185 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310188 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310190 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310193 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:29:38.316249 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310195 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310199 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310201 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310203 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310206 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310209 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310212 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310214 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310217 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310219 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310222 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310225 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310227 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310230 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310232 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310235 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310237 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310240 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310243 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:29:38.318587 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310245 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310248 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310250 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310253 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310256 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310258 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310261 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310264 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310266 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310269 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310271 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310274 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310276 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310278 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310281 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310284 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310286 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310289 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310291 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:29:38.319334 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310294 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310297 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310299 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310302 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310304 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310308 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310311 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310315 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310318 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310320 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310323 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310326 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310329 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310331 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310334 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310337 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310339 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310343 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310346 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:29:38.322033 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310349 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310352 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310355 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310357 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310360 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310363 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310366 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310369 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310372 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310374 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310377 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310380 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310382 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310387 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310389 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310392 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310394 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310398 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310412 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310414 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:29:38.322844 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310418 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310421 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310423 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310426 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310428 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.310431 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310502 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310512 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310520 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310526 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310532 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310537 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310541 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310545 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310549 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310552 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310555 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310558 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310562 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310565 2583 flags.go:64] FLAG: --cgroup-root="" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310580 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310583 2583 flags.go:64] FLAG: --client-ca-file="" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310587 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310590 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 21:29:38.423137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310595 2583 flags.go:64] FLAG: --cluster-domain="" Jul 13 21:29:38.378426 ip-10-0-133-19 systemd[1]: Started Kubernetes Kubelet. Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310599 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310602 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310605 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310609 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310612 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310615 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310618 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310621 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310624 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310627 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310630 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310632 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310635 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310639 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310642 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310645 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310648 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310651 2583 flags.go:64] FLAG: --enable-server="true" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310653 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310657 2583 flags.go:64] FLAG: --event-burst="100" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310660 2583 flags.go:64] FLAG: --event-qps="50" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310663 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310667 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 21:29:38.424235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310670 2583 flags.go:64] FLAG: --eviction-hard="" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310673 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310676 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310679 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310682 2583 flags.go:64] FLAG: --eviction-soft="" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310685 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310687 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310691 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310693 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310697 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310700 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310703 2583 flags.go:64] FLAG: --feature-gates="" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310707 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310710 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310713 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310715 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310718 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310721 2583 flags.go:64] FLAG: --help="false" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310724 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-133-19.ec2.internal" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310727 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310730 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310733 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310736 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 21:29:38.424998 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310740 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310742 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310745 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310748 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310751 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310754 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310757 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310759 2583 flags.go:64] FLAG: --kube-reserved="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310762 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310765 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310768 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310771 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310774 2583 flags.go:64] FLAG: --lock-file="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310776 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310779 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310782 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310787 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310790 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310793 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310796 2583 flags.go:64] FLAG: --logging-format="text" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310799 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310802 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310805 2583 flags.go:64] FLAG: --manifest-url="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310807 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310811 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 21:29:38.425774 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310814 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310818 2583 flags.go:64] FLAG: --max-pods="110" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310821 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310824 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310826 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310829 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310832 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310835 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310838 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310845 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310848 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310851 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310854 2583 flags.go:64] FLAG: --pod-cidr="" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310857 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310862 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310865 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310868 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310871 2583 flags.go:64] FLAG: --port="10250" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310874 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310877 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fa70d0490314670d" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310880 2583 flags.go:64] FLAG: --qos-reserved="" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310883 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310887 2583 flags.go:64] FLAG: --register-node="true" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310890 2583 flags.go:64] FLAG: --register-with-taints="" Jul 13 21:29:38.426564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310893 2583 flags.go:64] FLAG: --registry-burst="10" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310896 2583 flags.go:64] FLAG: --registry-qps="5" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310900 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310903 2583 flags.go:64] FLAG: --reserved-memory="" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310906 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310909 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310912 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310915 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310917 2583 flags.go:64] FLAG: --runonce="false" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310920 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310923 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310926 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310929 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310932 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310934 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310937 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310940 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310943 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310946 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310948 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310951 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310954 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310957 2583 flags.go:64] FLAG: --system-cgroups="" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310959 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310964 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 21:29:38.428508 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310967 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310970 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310974 2583 flags.go:64] FLAG: --tls-min-version="" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310977 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310980 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.310998 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.311002 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.311005 2583 flags.go:64] FLAG: --v="2" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.311009 2583 flags.go:64] FLAG: --version="false" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.311014 2583 flags.go:64] FLAG: --vmodule="" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.311017 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.311020 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311122 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311125 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311128 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311131 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311133 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311136 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311138 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311141 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311144 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311146 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:29:38.429297 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311149 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311151 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311154 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311156 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311159 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311161 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311164 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311166 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311169 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311171 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311174 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311177 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311179 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311182 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311184 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311188 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311190 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311193 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311195 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311200 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:29:38.430095 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311203 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311205 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311208 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311210 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311213 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311216 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311218 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311220 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311223 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311227 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311230 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311233 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311236 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311239 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311242 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311244 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311247 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311250 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311253 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:29:38.430868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311257 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311260 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311263 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311265 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311268 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311271 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311274 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311276 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311280 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311282 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311284 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311287 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311290 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311293 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311296 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311298 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311301 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311303 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311306 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311308 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:29:38.432398 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311311 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311313 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311316 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311318 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311321 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311323 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311326 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311328 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311331 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311334 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311338 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311340 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311343 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311345 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311348 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311350 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311353 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311355 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311357 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311360 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:29:38.433183 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311365 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311368 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311370 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311373 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311377 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311379 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311382 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311384 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311387 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.311389 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.312383 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.318384 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.318412 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.319115 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 21:29:38.434772 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.319125 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319175 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319180 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319183 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319186 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319189 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319192 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319195 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319197 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319200 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319203 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319206 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319208 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319211 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319213 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319216 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319218 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319221 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319223 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:29:38.435567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319226 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319228 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319231 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319233 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319235 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319238 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319240 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319243 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319245 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319248 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319251 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319255 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319258 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319260 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319262 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319265 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319267 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319270 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319272 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319275 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:29:38.437117 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319277 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319279 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319282 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319284 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319287 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319289 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319291 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319294 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319297 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319299 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319301 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319304 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319307 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319309 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319312 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319314 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319316 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319319 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319322 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:29:38.437912 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319326 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319328 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319332 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319335 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319339 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319342 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319344 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319347 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319350 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319352 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319355 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319358 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319361 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319364 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319367 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319370 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319372 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319375 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319378 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:29:38.439860 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319384 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319387 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319389 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319392 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319394 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319397 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319399 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319402 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319404 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319406 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319409 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319411 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319414 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319416 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319418 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319421 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319423 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319426 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319428 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319431 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:29:38.440656 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319434 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319436 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319439 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.319444 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319541 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319545 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319547 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319550 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319553 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319557 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319561 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319564 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319567 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319570 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:29:38.441914 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319572 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319575 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319578 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319580 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319583 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319586 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319590 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319592 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319595 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319598 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319600 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319603 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319605 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319608 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319611 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319613 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319616 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319618 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319621 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319624 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:29:38.442675 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319626 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319629 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319631 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319635 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319638 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319644 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319647 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319650 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319652 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319655 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319658 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319660 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319662 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319665 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319667 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319670 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319672 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319675 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319677 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:29:38.445002 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319679 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319682 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319684 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319687 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319690 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319692 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319695 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319697 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319699 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319702 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319704 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319707 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319710 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319712 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319714 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319717 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319720 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319722 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319725 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319728 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:29:38.445751 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319730 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319734 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319737 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319739 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319742 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319744 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319746 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319749 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319751 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319754 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319756 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319758 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319761 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319763 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319766 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319768 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319770 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319773 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319775 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319778 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:29:38.446567 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319780 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319782 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319785 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319787 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319789 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319792 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319794 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319797 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319799 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:38.319802 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.319807 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.319890 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.321849 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.323485 2583 server.go:1019] "Starting client certificate rotation" Jul 13 21:29:38.447342 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.323575 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.323605 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.327873 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.333007 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.345364 2583 log.go:25] "Validated CRI v1 runtime API" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.347676 2583 log.go:25] "Validated CRI v1 image API" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.348752 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.351050 2583 fs.go:135] Filesystem UUIDs: map[05f86d67-152d-4a8b-bffe-1d1db54395e0:/dev/nvme0n1p4 3b2af3df-3127-42d5-9c6b-3f7591560501:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 13 21:29:38.448490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.351075 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.358086 2583 manager.go:216] Machine: {Timestamp:2026-07-13 21:29:38.355771724 +0000 UTC m=+0.316067388 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3134169 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec200bd38fbce09534f2d02b4bdd1024 SystemUUID:ec200bd3-8fbc-e095-34f2-d02b4bdd1024 BootID:cdfd5901-8696-4e38-8f41-d4063870c8e3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:62:3b:25:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:62:3b:25:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:af:bf:f0:73:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.358219 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.358308 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.361153 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.361187 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-19.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.361977 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.362003 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.362013 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.362686 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.363929 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.366780 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.366914 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.367740 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.367761 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.367774 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.367785 2583 kubelet.go:398] "Adding apiserver pod source" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.367798 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 21:29:38.450068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.368170 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.368191 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.370270 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372150 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372176 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372313 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372324 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372331 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372336 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372343 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372348 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372798 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372804 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372809 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372815 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372828 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.372835 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.373567 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.373575 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.377706 2583 server.go:1268] "Started kubelet" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.377799 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.377888 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.378234 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.379280 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.380149 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.380573 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-19.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.380654 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.380732 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.383418 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.383584 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-19.ec2.internal.18c1f6b3893ff227 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-19.ec2.internal,UID:ip-10-0-133-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-19.ec2.internal,},FirstTimestamp:2026-07-13 21:29:38.377675303 +0000 UTC m=+0.337970947,LastTimestamp:2026-07-13 21:29:38.377675303 +0000 UTC m=+0.337970947,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-19.ec2.internal,}" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.384714 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.384729 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.384778 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385355 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385355 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385389 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385480 2583 factory.go:55] Registering systemd factory Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385489 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385493 2583 factory.go:223] Registration of the systemd container factory successfully Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385496 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385688 2583 factory.go:153] Registering CRI-O factory Jul 13 21:29:38.450859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385702 2583 factory.go:223] Registration of the crio container factory successfully Jul 13 21:29:38.452750 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385768 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 21:29:38.452750 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385786 2583 factory.go:103] Registering Raw factory Jul 13 21:29:38.452750 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.385798 2583 manager.go:1196] Started watching for new ooms in manager Jul 13 21:29:38.452750 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:38.386188 2583 manager.go:349] Starting recovery of all containers Jul 13 21:29:38.452750 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.394236 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 21:29:38.452750 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.394335 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 21:29:38.546646 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.485446 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:39.801591 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.585721 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:39.801591 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:38.605100 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 13 21:29:39.917733 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:39.827213 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:39.931358 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:39.927571 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.039180 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.028198 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.127715 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.122687 2583 manager.go:354] Recovery completed Jul 13 21:29:40.127715 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.124508 2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 13 21:29:40.128351 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.128031 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.129258 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.130459 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.130492 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.130507 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.131089 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.131148 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.131159 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.131158 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 21:29:40.132457 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.131188 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 13 21:29:40.135001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.132701 2583 policy_none.go:49] "None policy: Start" Jul 13 21:29:40.135001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.132717 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 21:29:40.135001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.132733 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 21:29:40.135001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.133726 2583 policy_none.go:47] "Start" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.154158 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.154183 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.154199 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.154234 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sbvml" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.154208 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-19.ec2.internal.18c1f6b3f1b9923f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-19.ec2.internal,UID:ip-10-0-133-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-19.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-19.ec2.internal,},FirstTimestamp:2026-07-13 21:29:40.130476607 +0000 UTC m=+0.949986417,LastTimestamp:2026-07-13 21:29:40.130476607 +0000 UTC m=+0.949986417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-19.ec2.internal,}" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.155521 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sbvml" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.165878 2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-19.ec2.internal\" not found" node="ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193251 2583 manager.go:337] "Starting Device Plugin manager" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.193275 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193286 2583 server.go:87] "Starting device plugin registration server" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193477 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193487 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193588 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193659 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.193667 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.195312 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 21:29:40.248402 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.196159 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.293976 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.293960 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.294749 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.294722 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.294860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.294754 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.294860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.294765 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.294860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.294806 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.304009 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.303994 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.304067 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.304013 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-19.ec2.internal\": node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.319032 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.319017 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.344117 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.344091 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 21:29:40.345385 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.345366 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 21:29:40.345470 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.345397 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 21:29:40.345470 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.345413 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 21:29:40.345470 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.345442 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 21:29:40.347382 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.347368 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:29:40.419104 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.419053 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.446475 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.446449 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal"] Jul 13 21:29:40.446840 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.446525 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.447256 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.447242 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.447318 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.447266 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.447318 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.447281 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.449196 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449184 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.449316 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449302 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.449396 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449329 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.449855 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449838 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.449929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449861 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.449929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449870 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.449929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449895 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.449929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449913 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.449929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.449922 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.451903 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.451889 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.451987 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.451911 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:29:40.452498 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.452484 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:29:40.452564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.452512 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:29:40.452564 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.452525 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:29:40.464373 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.464360 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 21:29:40.464492 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.464470 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 21:29:40.464591 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.464527 2583 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aaaf2c34929e14f038d429880e377eb7-016897258eb91c10.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-133-19.ec2.internal.18c1f6b3f1b9e8c4\": read tcp 10.0.133.19:38556->34.198.103.203:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-133-19.ec2.internal.18c1f6b3f1b9e8c4 default 6294 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-19.ec2.internal,UID:ip-10-0-133-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-19.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-19.ec2.internal,},FirstTimestamp:2026-07-13 21:29:40 +0000 UTC,LastTimestamp:2026-07-13 21:29:40.447273067 +0000 UTC m=+1.266782867,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-19.ec2.internal,}" Jul 13 21:29:40.475313 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.475299 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-19.ec2.internal\" not found" node="ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.478945 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.478932 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-19.ec2.internal\" not found" node="ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.519157 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.519141 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.537194 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.537175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/427b4e67420246f4f9a4a3e967ae607f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal\" (UID: \"427b4e67420246f4f9a4a3e967ae607f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.537277 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.537198 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/427b4e67420246f4f9a4a3e967ae607f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal\" (UID: \"427b4e67420246f4f9a4a3e967ae607f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.537277 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.537214 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/052647b0243d11ac5a4990de0eb190ae-config\") pod \"kube-apiserver-proxy-ip-10-0-133-19.ec2.internal\" (UID: \"052647b0243d11ac5a4990de0eb190ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.619645 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.619627 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.638041 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.638023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/427b4e67420246f4f9a4a3e967ae607f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal\" (UID: \"427b4e67420246f4f9a4a3e967ae607f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.638103 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.638044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/052647b0243d11ac5a4990de0eb190ae-config\") pod \"kube-apiserver-proxy-ip-10-0-133-19.ec2.internal\" (UID: \"052647b0243d11ac5a4990de0eb190ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.638103 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.638059 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/427b4e67420246f4f9a4a3e967ae607f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal\" (UID: \"427b4e67420246f4f9a4a3e967ae607f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.638103 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.638082 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/427b4e67420246f4f9a4a3e967ae607f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal\" (UID: \"427b4e67420246f4f9a4a3e967ae607f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.638214 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.638123 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/052647b0243d11ac5a4990de0eb190ae-config\") pod \"kube-apiserver-proxy-ip-10-0-133-19.ec2.internal\" (UID: \"052647b0243d11ac5a4990de0eb190ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.638214 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.638121 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/427b4e67420246f4f9a4a3e967ae607f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal\" (UID: \"427b4e67420246f4f9a4a3e967ae607f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.706277 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.706238 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:29:40.720263 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.720246 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.777686 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.777661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.782244 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.782227 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" Jul 13 21:29:40.820321 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.820299 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.849783 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.849754 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:29:40.920793 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:40.920760 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-19.ec2.internal\" not found" Jul 13 21:29:40.975522 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:40.975477 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:29:41.026340 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.026320 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" Jul 13 21:29:41.036206 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.036189 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 21:29:41.037161 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.037151 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" Jul 13 21:29:41.045317 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.045305 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 21:29:41.158429 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.158397 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:24:40 +0000 UTC" deadline="2028-01-28 09:49:09.684521542 +0000 UTC" Jul 13 21:29:41.158429 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.158423 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13524h19m28.526102438s" Jul 13 21:29:41.346695 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.346660 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427b4e67420246f4f9a4a3e967ae607f.slice/crio-ac2b65d5eea7bff9d8afbdbc324e27e723168acc82f57c7443f8949081d9484e WatchSource:0}: Error finding container ac2b65d5eea7bff9d8afbdbc324e27e723168acc82f57c7443f8949081d9484e: Status 404 returned error can't find the container with id ac2b65d5eea7bff9d8afbdbc324e27e723168acc82f57c7443f8949081d9484e Jul 13 21:29:41.347332 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.347312 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052647b0243d11ac5a4990de0eb190ae.slice/crio-bed9c34faf2b46fc0cf276e1d021d05f750a5954d8971b2c3989a4e1860aaa58 WatchSource:0}: Error finding container bed9c34faf2b46fc0cf276e1d021d05f750a5954d8971b2c3989a4e1860aaa58: Status 404 returned error can't find the container with id bed9c34faf2b46fc0cf276e1d021d05f750a5954d8971b2c3989a4e1860aaa58 Jul 13 21:29:41.352523 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.352509 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:29:41.509354 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.509335 2583 apiserver.go:52] "Watching apiserver" Jul 13 21:29:41.518490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.518465 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 21:29:41.518820 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.518799 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-pppts","openshift-ovn-kubernetes/ovnkube-node-wb7hh","kube-system/konnectivity-agent-tklwp","openshift-dns/node-resolver-vhlcp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal","openshift-multus/multus-additional-cni-plugins-gb9zp","openshift-multus/multus-bd4p9","openshift-multus/network-metrics-daemon-49ngc","kube-system/global-pull-secret-syncer-ckzb9","kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq","openshift-cluster-node-tuning-operator/tuned-qbrv2","openshift-image-registry/node-ca-dlbfr","openshift-network-diagnostics/network-check-target-bcl7j"] Jul 13 21:29:41.522482 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.522468 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.524606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.524588 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 21:29:41.524825 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.524804 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.524825 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.524820 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.524969 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.524820 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 21:29:41.524969 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.524894 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bc9kb\"" Jul 13 21:29:41.525262 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.525246 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.526126 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.526106 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 21:29:41.527389 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.527358 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 21:29:41.527389 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.527362 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-smdsh\"" Jul 13 21:29:41.527548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.527409 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.527548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.527414 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.527548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.527470 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.529638 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.529588 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-llx9z\"" Jul 13 21:29:41.529638 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.529611 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 21:29:41.529762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.529669 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 21:29:41.530388 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.530372 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.532314 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.532297 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.532501 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.532485 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sjfbl\"" Jul 13 21:29:41.532574 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.532514 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:41.532574 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.532488 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.532574 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.532564 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:41.534662 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.534645 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.536798 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.536761 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 21:29:41.536884 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.536817 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.536884 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.536859 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 21:29:41.536983 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.536857 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:41.536983 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.536953 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8pp7f\"" Jul 13 21:29:41.537817 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.537801 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 21:29:41.539080 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.539046 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.540428 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-conf-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540505 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-etc-kubernetes\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540505 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8e4813b-1496-4f6e-bc32-5d875132c8cb-konnectivity-ca\") pod \"konnectivity-agent-tklwp\" (UID: \"f8e4813b-1496-4f6e-bc32-5d875132c8cb\") " pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.540505 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540465 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-socket-dir-parent\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540505 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540496 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-socket-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540515 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8e4813b-1496-4f6e-bc32-5d875132c8cb-agent-certs\") pod \"konnectivity-agent-tklwp\" (UID: \"f8e4813b-1496-4f6e-bc32-5d875132c8cb\") " pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540530 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-cnibin\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-os-release\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540562 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-k8s-cni-cncf-io\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540623 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.540649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540644 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzv2b\" (UniqueName: \"kubernetes.io/projected/d7f570a4-0f69-4f87-9aa1-161c25c793ac-kube-api-access-mzv2b\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-cni-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-cni-bin\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540706 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-cni-multus\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-kubelet\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540767 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-device-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540853 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-system-cni-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6131f25-306d-4070-baff-be122381d9c4-multus-daemon-config\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.540934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540901 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-multus-certs\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540944 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-registration-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.540966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-sys-fs\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvndd\" (UniqueName: \"kubernetes.io/projected/e6131f25-306d-4070-baff-be122381d9c4-kube-api-access-bvndd\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541045 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6131f25-306d-4070-baff-be122381d9c4-cni-binary-copy\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541087 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-netns\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541111 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-hostroot\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541240 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541248 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 21:29:41.541384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541248 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nn45c\"" Jul 13 21:29:41.541860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541473 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.541860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.541690 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.543853 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.543837 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:41.543931 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.543884 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:41.545003 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.544968 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 21:29:41.545080 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.545030 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 21:29:41.545080 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.545040 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 21:29:41.545192 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.545074 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p2qx7\"" Jul 13 21:29:41.545192 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.545030 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.545192 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.545119 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 21:29:41.545192 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.545078 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.546430 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.546408 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.549177 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.549159 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 21:29:41.549627 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.549607 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5gb6l\"" Jul 13 21:29:41.550032 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.549869 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.550032 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.549954 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.550450 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.550321 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.552373 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.552358 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 21:29:41.552520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.552504 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5bhx2\"" Jul 13 21:29:41.552594 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.552524 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:29:41.558732 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.558671 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mmsm2" Jul 13 21:29:41.566342 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.566324 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mmsm2" Jul 13 21:29:41.575208 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.575193 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:29:41.627507 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.627469 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 21:29:41.641947 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.641928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-socket-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.642054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.641954 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8e4813b-1496-4f6e-bc32-5d875132c8cb-agent-certs\") pod \"konnectivity-agent-tklwp\" (UID: \"f8e4813b-1496-4f6e-bc32-5d875132c8cb\") " pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.642054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.641977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.642054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-systemd-units\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642024 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysconfig\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642075 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642106 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-socket-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642106 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kttl\" (UniqueName: \"kubernetes.io/projected/5afe6508-8b7b-4083-ac75-e95e48666e82-kube-api-access-9kttl\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642143 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-os-release\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642160 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642178 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzv2b\" (UniqueName: \"kubernetes.io/projected/d7f570a4-0f69-4f87-9aa1-161c25c793ac-kube-api-access-mzv2b\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-cni-binary-copy\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642228 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-os-release\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642236 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-run-netns\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642274 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-systemd\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.642296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642295 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-lib-modules\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642328 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-cni-bin\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642334 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642364 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-cni-bin\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-kubelet\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642366 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-kubelet\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f589797-94ec-4487-b525-157b6547a3d5-dbus\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6732ff06-54b6-4879-9977-4bc85ce107da-serviceca\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642422 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-var-lib-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysctl-d\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642462 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-tuned\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642488 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-system-cni-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642503 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f589797-94ec-4487-b525-157b6547a3d5-kubelet-config\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642530 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-node-log\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642570 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-env-overrides\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642591 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-kubelet\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.642729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642603 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-system-cni-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642614 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-ovn\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-modprobe-d\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642662 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c58339a5-68e2-4182-935e-8b29cf2346a6-tmp\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvndd\" (UniqueName: \"kubernetes.io/projected/e6131f25-306d-4070-baff-be122381d9c4-kube-api-access-bvndd\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642714 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-os-release\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642782 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysctl-conf\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-var-lib-kubelet\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6131f25-306d-4070-baff-be122381d9c4-cni-binary-copy\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-netns\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642866 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-hostroot\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8e4813b-1496-4f6e-bc32-5d875132c8cb-konnectivity-ca\") pod \"konnectivity-agent-tklwp\" (UID: \"f8e4813b-1496-4f6e-bc32-5d875132c8cb\") " pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642920 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642946 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c18c033c-9b29-4d74-bede-533cac01c90b-host-slash\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642946 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-hostroot\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.642977 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-netns\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.643371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-system-cni-dir\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643045 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-socket-dir-parent\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-slash\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643115 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-cnibin\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-k8s-cni-cncf-io\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643201 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-cnibin\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643215 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-k8s-cni-cncf-io\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643208 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-socket-dir-parent\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjjzc\" (UniqueName: \"kubernetes.io/projected/6732ff06-54b6-4879-9977-4bc85ce107da-kube-api-access-xjjzc\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643212 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-cnibin\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643264 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-cni-bin\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643291 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643301 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643314 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-ovnkube-config\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643330 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6131f25-306d-4070-baff-be122381d9c4-cni-binary-copy\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-cni-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643392 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-cni-multus\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-device-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643428 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-var-lib-cni-multus\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643434 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bts4h\" (UniqueName: \"kubernetes.io/projected/f6839532-b922-48be-8590-c4dab166cc5a-kube-api-access-bts4h\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643392 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8e4813b-1496-4f6e-bc32-5d875132c8cb-konnectivity-ca\") pod \"konnectivity-agent-tklwp\" (UID: \"f8e4813b-1496-4f6e-bc32-5d875132c8cb\") " pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643479 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-cni-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643486 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-device-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-host\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643547 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q69r\" (UniqueName: \"kubernetes.io/projected/c58339a5-68e2-4182-935e-8b29cf2346a6-kube-api-access-4q69r\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6131f25-306d-4070-baff-be122381d9c4-multus-daemon-config\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-multus-certs\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643622 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-registration-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-sys-fs\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643668 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6732ff06-54b6-4879-9977-4bc85ce107da-host\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643672 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-host-run-multus-certs\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-registration-dir\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643699 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-systemd\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.644584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643708 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7f570a4-0f69-4f87-9aa1-161c25c793ac-sys-fs\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643723 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-etc-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643743 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-sys\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-tmp-dir\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643818 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp5z4\" (UniqueName: \"kubernetes.io/projected/f688fd0b-cb34-4172-8905-92a5f9a081db-kube-api-access-sp5z4\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643857 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-ovnkube-script-lib\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643889 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-run\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fns54\" (UniqueName: \"kubernetes.io/projected/c18c033c-9b29-4d74-bede-533cac01c90b-kube-api-access-fns54\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643936 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-hosts-file\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643964 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-conf-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.643989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-etc-kubernetes\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644014 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6131f25-306d-4070-baff-be122381d9c4-multus-daemon-config\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644035 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-etc-kubernetes\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-log-socket\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644059 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6131f25-306d-4070-baff-be122381d9c4-multus-conf-dir\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644095 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-cni-netd\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.645053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644120 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6839532-b922-48be-8590-c4dab166cc5a-ovn-node-metrics-cert\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.645592 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:41.645592 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644174 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kftbv\" (UniqueName: \"kubernetes.io/projected/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-kube-api-access-kftbv\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.645592 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-kubernetes\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.645592 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.644218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c18c033c-9b29-4d74-bede-533cac01c90b-iptables-alerter-script\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.645817 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.645799 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8e4813b-1496-4f6e-bc32-5d875132c8cb-agent-certs\") pod \"konnectivity-agent-tklwp\" (UID: \"f8e4813b-1496-4f6e-bc32-5d875132c8cb\") " pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.654744 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.654717 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzv2b\" (UniqueName: \"kubernetes.io/projected/d7f570a4-0f69-4f87-9aa1-161c25c793ac-kube-api-access-mzv2b\") pod \"aws-ebs-csi-driver-node-jqpzq\" (UID: \"d7f570a4-0f69-4f87-9aa1-161c25c793ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.655290 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.655271 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvndd\" (UniqueName: \"kubernetes.io/projected/e6131f25-306d-4070-baff-be122381d9c4-kube-api-access-bvndd\") pod \"multus-bd4p9\" (UID: \"e6131f25-306d-4070-baff-be122381d9c4\") " pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.745260 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-os-release\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745267 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745283 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysctl-conf\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.745380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745302 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-var-lib-kubelet\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.745380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745327 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745343 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c18c033c-9b29-4d74-bede-533cac01c90b-host-slash\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.745380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-system-cni-dir\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-slash\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-cnibin\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745449 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjjzc\" (UniqueName: \"kubernetes.io/projected/6732ff06-54b6-4879-9977-4bc85ce107da-kube-api-access-xjjzc\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745504 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-var-lib-kubelet\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745514 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-system-cni-dir\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745561 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c18c033c-9b29-4d74-bede-533cac01c90b-host-slash\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-os-release\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745606 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-slash\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745638 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-cnibin\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.745670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745646 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-cni-bin\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745680 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-cni-bin\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-ovnkube-config\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745720 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745688 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysctl-conf\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745734 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bts4h\" (UniqueName: \"kubernetes.io/projected/f6839532-b922-48be-8590-c4dab166cc5a-kube-api-access-bts4h\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-host\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745806 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q69r\" (UniqueName: \"kubernetes.io/projected/c58339a5-68e2-4182-935e-8b29cf2346a6-kube-api-access-4q69r\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6732ff06-54b6-4879-9977-4bc85ce107da-host\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-systemd\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745875 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-etc-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745897 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-sys\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-tmp-dir\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745939 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp5z4\" (UniqueName: \"kubernetes.io/projected/f688fd0b-cb34-4172-8905-92a5f9a081db-kube-api-access-sp5z4\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745944 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745968 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-ovnkube-script-lib\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.746223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745973 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-systemd\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.745946 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-run\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746041 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-sys\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746057 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-host\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746085 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-run\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746094 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6732ff06-54b6-4879-9977-4bc85ce107da-host\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fns54\" (UniqueName: \"kubernetes.io/projected/c18c033c-9b29-4d74-bede-533cac01c90b-kube-api-access-fns54\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-hosts-file\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746160 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-log-socket\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746182 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746204 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-cni-netd\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6839532-b922-48be-8590-c4dab166cc5a-ovn-node-metrics-cert\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746251 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746261 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-tmp-dir\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746274 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kftbv\" (UniqueName: \"kubernetes.io/projected/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-kube-api-access-kftbv\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746278 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-ovnkube-config\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746280 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-log-socket\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747012 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746018 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-etc-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746334 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-cni-netd\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746344 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.746390 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746416 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-hosts-file\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-kubernetes\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.746452 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:42.246433577 +0000 UTC m=+3.065943377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c18c033c-9b29-4d74-bede-533cac01c90b-iptables-alerter-script\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746491 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-kubernetes\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746505 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746529 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-systemd-units\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746554 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysconfig\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746594 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-ovnkube-script-lib\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746601 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kttl\" (UniqueName: \"kubernetes.io/projected/5afe6508-8b7b-4083-ac75-e95e48666e82-kube-api-access-9kttl\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-cni-binary-copy\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.747896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746673 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f688fd0b-cb34-4172-8905-92a5f9a081db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746681 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-systemd-units\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746676 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-run-netns\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746715 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-systemd\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-lib-modules\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f589797-94ec-4487-b525-157b6547a3d5-dbus\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6732ff06-54b6-4879-9977-4bc85ce107da-serviceca\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-var-lib-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysctl-d\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746915 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysconfig\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746931 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-systemd\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746957 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-run-netns\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-tuned\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.746976 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-sysctl-d\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747003 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f589797-94ec-4487-b525-157b6547a3d5-kubelet-config\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747024 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-var-lib-openvswitch\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.748648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747054 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c18c033c-9b29-4d74-bede-533cac01c90b-iptables-alerter-script\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747008 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f589797-94ec-4487-b525-157b6547a3d5-dbus\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747070 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f589797-94ec-4487-b525-157b6547a3d5-kubelet-config\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747101 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-lib-modules\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747059 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-node-log\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f688fd0b-cb34-4172-8905-92a5f9a081db-cni-binary-copy\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747144 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-env-overrides\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-node-log\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.747159 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-kubelet\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747204 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-ovn\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.747229 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:42.247211336 +0000 UTC m=+3.066721128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747235 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-run-ovn\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-modprobe-d\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747269 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6839532-b922-48be-8590-c4dab166cc5a-host-kubelet\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c58339a5-68e2-4182-935e-8b29cf2346a6-tmp\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.749364 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747317 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6732ff06-54b6-4879-9977-4bc85ce107da-serviceca\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.749894 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747346 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-modprobe-d\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.749894 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.747501 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6839532-b922-48be-8590-c4dab166cc5a-env-overrides\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749894 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.748595 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6839532-b922-48be-8590-c4dab166cc5a-ovn-node-metrics-cert\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.749894 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.749114 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c58339a5-68e2-4182-935e-8b29cf2346a6-etc-tuned\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.749894 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.749347 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c58339a5-68e2-4182-935e-8b29cf2346a6-tmp\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.753392 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.753374 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bts4h\" (UniqueName: \"kubernetes.io/projected/f6839532-b922-48be-8590-c4dab166cc5a-kube-api-access-bts4h\") pod \"ovnkube-node-wb7hh\" (UID: \"f6839532-b922-48be-8590-c4dab166cc5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.753478 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.753457 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjjzc\" (UniqueName: \"kubernetes.io/projected/6732ff06-54b6-4879-9977-4bc85ce107da-kube-api-access-xjjzc\") pod \"node-ca-dlbfr\" (UID: \"6732ff06-54b6-4879-9977-4bc85ce107da\") " pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.757839 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.757822 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:29:41.757928 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.757846 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:29:41.757928 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.757857 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:41.757928 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:41.757911 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:42.257895216 +0000 UTC m=+3.077405027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:41.760040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.759985 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kftbv\" (UniqueName: \"kubernetes.io/projected/b0438b82-c3d6-4ce8-8d45-fb3ac41c659c-kube-api-access-kftbv\") pod \"node-resolver-vhlcp\" (UID: \"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c\") " pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.760406 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.760362 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kttl\" (UniqueName: \"kubernetes.io/projected/5afe6508-8b7b-4083-ac75-e95e48666e82-kube-api-access-9kttl\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:41.760711 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.760684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q69r\" (UniqueName: \"kubernetes.io/projected/c58339a5-68e2-4182-935e-8b29cf2346a6-kube-api-access-4q69r\") pod \"tuned-qbrv2\" (UID: \"c58339a5-68e2-4182-935e-8b29cf2346a6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.760789 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.760686 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fns54\" (UniqueName: \"kubernetes.io/projected/c18c033c-9b29-4d74-bede-533cac01c90b-kube-api-access-fns54\") pod \"iptables-alerter-pppts\" (UID: \"c18c033c-9b29-4d74-bede-533cac01c90b\") " pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.760789 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.760688 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp5z4\" (UniqueName: \"kubernetes.io/projected/f688fd0b-cb34-4172-8905-92a5f9a081db-kube-api-access-sp5z4\") pod \"multus-additional-cni-plugins-gb9zp\" (UID: \"f688fd0b-cb34-4172-8905-92a5f9a081db\") " pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.831906 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.831883 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bd4p9" Jul 13 21:29:41.837783 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.837750 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" Jul 13 21:29:41.840839 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.840819 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6131f25_306d_4070_baff_be122381d9c4.slice/crio-c50b4c2d086363a547c08f1e67802e9efa7c3573539f756f8d1d16e69ed1e8b5 WatchSource:0}: Error finding container c50b4c2d086363a547c08f1e67802e9efa7c3573539f756f8d1d16e69ed1e8b5: Status 404 returned error can't find the container with id c50b4c2d086363a547c08f1e67802e9efa7c3573539f756f8d1d16e69ed1e8b5 Jul 13 21:29:41.845395 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.845376 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:29:41.847507 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.847446 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f570a4_0f69_4f87_9aa1_161c25c793ac.slice/crio-e066382418cdc9dd2b8a2f7cd9ae19afdb2cea14bebd4ecbdfe94cff46cfdbd5 WatchSource:0}: Error finding container e066382418cdc9dd2b8a2f7cd9ae19afdb2cea14bebd4ecbdfe94cff46cfdbd5: Status 404 returned error can't find the container with id e066382418cdc9dd2b8a2f7cd9ae19afdb2cea14bebd4ecbdfe94cff46cfdbd5 Jul 13 21:29:41.851467 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.851446 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhlcp" Jul 13 21:29:41.855653 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.855635 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" Jul 13 21:29:41.856309 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.856291 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e4813b_1496_4f6e_bc32_5d875132c8cb.slice/crio-4aa1c9288b05fffebc10f9194d56a463ea0f33319d150e446196ec8dfbef906d WatchSource:0}: Error finding container 4aa1c9288b05fffebc10f9194d56a463ea0f33319d150e446196ec8dfbef906d: Status 404 returned error can't find the container with id 4aa1c9288b05fffebc10f9194d56a463ea0f33319d150e446196ec8dfbef906d Jul 13 21:29:41.860974 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.860961 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dlbfr" Jul 13 21:29:41.865315 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.865290 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0438b82_c3d6_4ce8_8d45_fb3ac41c659c.slice/crio-c55a5910f039511f9f243866a7b97f4dc6ddc0ad448cb6a4279c68e30fbce658 WatchSource:0}: Error finding container c55a5910f039511f9f243866a7b97f4dc6ddc0ad448cb6a4279c68e30fbce658: Status 404 returned error can't find the container with id c55a5910f039511f9f243866a7b97f4dc6ddc0ad448cb6a4279c68e30fbce658 Jul 13 21:29:41.865873 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.865705 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:29:41.871659 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.871516 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pppts" Jul 13 21:29:41.872176 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.872148 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6732ff06_54b6_4879_9977_4bc85ce107da.slice/crio-88eeac2fbcf99234fab8c574158bb97194773d157ee2cfd7279313a68f550788 WatchSource:0}: Error finding container 88eeac2fbcf99234fab8c574158bb97194773d157ee2cfd7279313a68f550788: Status 404 returned error can't find the container with id 88eeac2fbcf99234fab8c574158bb97194773d157ee2cfd7279313a68f550788 Jul 13 21:29:41.875582 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:41.875434 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" Jul 13 21:29:41.877856 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.877589 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6839532_b922_48be_8590_c4dab166cc5a.slice/crio-09aa8c411a8086c14a1d497c0b06d9d6735782a4beaea0e649f09c3d8f0b05d2 WatchSource:0}: Error finding container 09aa8c411a8086c14a1d497c0b06d9d6735782a4beaea0e649f09c3d8f0b05d2: Status 404 returned error can't find the container with id 09aa8c411a8086c14a1d497c0b06d9d6735782a4beaea0e649f09c3d8f0b05d2 Jul 13 21:29:41.888195 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:29:41.888172 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58339a5_68e2_4182_935e_8b29cf2346a6.slice/crio-6f685793c645803dd5ff752b270230cdab6caccec3e33d55b7f2b2f2f8e40a68 WatchSource:0}: Error finding container 6f685793c645803dd5ff752b270230cdab6caccec3e33d55b7f2b2f2f8e40a68: Status 404 returned error can't find the container with id 6f685793c645803dd5ff752b270230cdab6caccec3e33d55b7f2b2f2f8e40a68 Jul 13 21:29:42.251046 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.250969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:42.251046 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.251032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:42.251261 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.251132 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:42.251261 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.251179 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:42.251261 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.251211 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:43.251177738 +0000 UTC m=+4.070687544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:42.251261 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.251241 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:43.251225803 +0000 UTC m=+4.070735590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:42.349958 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.349901 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" event={"ID":"c58339a5-68e2-4182-935e-8b29cf2346a6","Type":"ContainerStarted","Data":"6f685793c645803dd5ff752b270230cdab6caccec3e33d55b7f2b2f2f8e40a68"} Jul 13 21:29:42.351320 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.351294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pppts" event={"ID":"c18c033c-9b29-4d74-bede-533cac01c90b","Type":"ContainerStarted","Data":"17adc9acd9376b264d6d9407c57f64993c4067d2a71b2069b23c7542c7160d28"} Jul 13 21:29:42.351438 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.351400 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:42.351558 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.351538 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:29:42.351558 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.351555 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:29:42.351700 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.351563 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:42.351700 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:42.351606 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:43.351591789 +0000 UTC m=+4.171101586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:42.352437 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.352406 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhlcp" event={"ID":"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c","Type":"ContainerStarted","Data":"c55a5910f039511f9f243866a7b97f4dc6ddc0ad448cb6a4279c68e30fbce658"} Jul 13 21:29:42.353636 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.353608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tklwp" event={"ID":"f8e4813b-1496-4f6e-bc32-5d875132c8cb","Type":"ContainerStarted","Data":"4aa1c9288b05fffebc10f9194d56a463ea0f33319d150e446196ec8dfbef906d"} Jul 13 21:29:42.354965 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.354907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" event={"ID":"d7f570a4-0f69-4f87-9aa1-161c25c793ac","Type":"ContainerStarted","Data":"e066382418cdc9dd2b8a2f7cd9ae19afdb2cea14bebd4ecbdfe94cff46cfdbd5"} Jul 13 21:29:42.356308 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.356060 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bd4p9" event={"ID":"e6131f25-306d-4070-baff-be122381d9c4","Type":"ContainerStarted","Data":"c50b4c2d086363a547c08f1e67802e9efa7c3573539f756f8d1d16e69ed1e8b5"} Jul 13 21:29:42.357235 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.357194 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" event={"ID":"427b4e67420246f4f9a4a3e967ae607f","Type":"ContainerStarted","Data":"ac2b65d5eea7bff9d8afbdbc324e27e723168acc82f57c7443f8949081d9484e"} Jul 13 21:29:42.358469 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.358450 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" event={"ID":"052647b0243d11ac5a4990de0eb190ae","Type":"ContainerStarted","Data":"bed9c34faf2b46fc0cf276e1d021d05f750a5954d8971b2c3989a4e1860aaa58"} Jul 13 21:29:42.359725 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.359703 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"09aa8c411a8086c14a1d497c0b06d9d6735782a4beaea0e649f09c3d8f0b05d2"} Jul 13 21:29:42.361002 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.360979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dlbfr" event={"ID":"6732ff06-54b6-4879-9977-4bc85ce107da","Type":"ContainerStarted","Data":"88eeac2fbcf99234fab8c574158bb97194773d157ee2cfd7279313a68f550788"} Jul 13 21:29:42.362283 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.362259 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerStarted","Data":"64b36ff1be7df7e29396ca8cb2cf6f755b36d2012d6b6afe639c24ebe8d42f10"} Jul 13 21:29:42.569294 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.569218 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:24:41 +0000 UTC" deadline="2028-03-20 21:46:54.262086396 +0000 UTC" Jul 13 21:29:42.569294 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:42.569255 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14784h17m11.692835377s" Jul 13 21:29:43.260345 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.259937 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:43.260345 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.260004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:43.260345 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.260155 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:43.260345 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.260210 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:45.260192999 +0000 UTC m=+6.079702791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:43.260345 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.260274 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:43.260345 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.260304 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:45.260293208 +0000 UTC m=+6.079802999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:43.346378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.345874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:43.346378 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.346022 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:43.346378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.346101 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:43.346378 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.346200 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:43.346378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.346258 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:43.346378 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.346324 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:43.360643 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.360342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:43.360643 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.360514 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:29:43.360643 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.360533 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:29:43.360643 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.360544 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:43.360643 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:43.360598 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:45.360577937 +0000 UTC m=+6.180087736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:43.570007 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.569869 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:24:41 +0000 UTC" deadline="2028-02-28 17:21:29.267041257 +0000 UTC" Jul 13 21:29:43.570007 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:43.569921 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14275h51m45.697123737s" Jul 13 21:29:45.275299 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:45.275264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:45.275753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:45.275337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:45.275753 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.275468 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:45.275753 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.275527 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:49.275506403 +0000 UTC m=+10.095016191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:45.275753 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.275593 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:45.275753 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.275634 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:49.275622161 +0000 UTC m=+10.095131952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:45.345879 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:45.345849 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:45.345879 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:45.345873 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:45.346119 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.345977 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:45.346119 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.346078 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:45.346228 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:45.346131 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:45.346228 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.346209 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:45.375628 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:45.375596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:45.375751 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.375715 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:29:45.375751 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.375727 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:29:45.375751 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.375735 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:45.375934 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:45.375784 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:49.375758748 +0000 UTC m=+10.195268535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:47.345706 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:47.345647 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:47.346241 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:47.345768 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:47.346241 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:47.345845 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:47.346241 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:47.345921 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:47.346241 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:47.346057 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:47.346241 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:47.346158 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:49.306308 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:49.306045 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:49.306308 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:49.306117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:49.306308 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.306236 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:49.306308 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.306255 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:49.306308 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.306306 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:57.306283253 +0000 UTC m=+18.125793055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:49.307117 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.306325 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:57.306315224 +0000 UTC m=+18.125825015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:49.345963 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:49.345931 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:49.346118 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.346071 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:49.346388 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:49.346369 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:49.346500 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.346460 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:49.346566 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:49.346525 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:49.346621 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.346608 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:49.406961 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:49.406876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:49.407081 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.407052 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:29:49.407081 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.407077 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:29:49.407210 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.407090 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:49.407286 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:49.407140 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:57.407122451 +0000 UTC m=+18.226632245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:51.346177 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:51.346151 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:51.346557 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:51.346193 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:51.346557 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:51.346283 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:51.346557 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:51.346322 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:51.346557 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:51.346346 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:51.346557 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:51.346454 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:53.345914 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:53.345703 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:53.346327 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:53.345703 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:53.346327 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:53.346031 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:53.346327 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:53.345703 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:53.346327 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:53.346091 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:53.346327 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:53.346174 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:55.346466 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:55.346434 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:55.346466 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:55.346456 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:55.346917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:55.346463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:55.346917 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:55.346546 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:55.346917 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:55.346602 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:55.346917 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:55.346714 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:57.346407 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.346330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:57.346407 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.346349 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:57.346407 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.346375 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:57.346948 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.346459 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:29:57.346948 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.346506 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:57.346948 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.346593 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:57.367289 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.367260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:57.367439 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.367320 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:57.367439 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.367435 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:57.367536 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.367437 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:57.367536 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.367487 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:13.367470303 +0000 UTC m=+34.186980093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:29:57.367536 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.367507 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:13.367498116 +0000 UTC m=+34.187007902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:29:57.390642 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.390616 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" event={"ID":"427b4e67420246f4f9a4a3e967ae607f","Type":"ContainerStarted","Data":"7ee586a9aaffdd106b89fb6a0a4da0b539cbd277d06d4aff835d691c5206464f"} Jul 13 21:29:57.468043 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:57.468016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:57.468198 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.468182 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:29:57.468248 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.468204 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:29:57.468248 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.468213 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:57.468314 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:57.468263 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:13.468245845 +0000 UTC m=+34.287755645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:29:59.346415 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:59.346384 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:29:59.346945 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:59.346502 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:29:59.346945 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:59.346509 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:29:59.346945 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:29:59.346538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:29:59.346945 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:59.346598 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:29:59.346945 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:29:59.346683 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:00.397207 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.397003 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerStarted","Data":"9c023c9d3ab6995cb55a3c3be6c97e2690903fd7dbcd1f24dea81c42ac1189da"} Jul 13 21:30:00.398743 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.398712 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" event={"ID":"c58339a5-68e2-4182-935e-8b29cf2346a6","Type":"ContainerStarted","Data":"65bb226276a995a2d1402b274fea21c8fd03f34ff9cce33eee6e760daa8af313"} Jul 13 21:30:00.400463 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.400431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhlcp" event={"ID":"b0438b82-c3d6-4ce8-8d45-fb3ac41c659c","Type":"ContainerStarted","Data":"1ac476fb3dd79fe013a688f68cf1b99f578739cec7eb3f46003a4ba5ff031f67"} Jul 13 21:30:00.401992 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.401949 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tklwp" event={"ID":"f8e4813b-1496-4f6e-bc32-5d875132c8cb","Type":"ContainerStarted","Data":"62b76dc68879692be717d2a090d67c72e19b606f6dfaa2d5a2aaca6e0f5c7731"} Jul 13 21:30:00.403696 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.403666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" event={"ID":"d7f570a4-0f69-4f87-9aa1-161c25c793ac","Type":"ContainerStarted","Data":"d3bd145a026f92b10912cf5e92e3c82e411e2ceeec74c31fdb753c9f0af646b2"} Jul 13 21:30:00.405237 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.405211 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bd4p9" event={"ID":"e6131f25-306d-4070-baff-be122381d9c4","Type":"ContainerStarted","Data":"3b33a0af82ea23ada6e6903f098a21c391b85bddfe51c168f96edca527a05f1a"} Jul 13 21:30:00.406609 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.406590 2583 generic.go:356] "Generic (PLEG): container finished" podID="427b4e67420246f4f9a4a3e967ae607f" containerID="7ee586a9aaffdd106b89fb6a0a4da0b539cbd277d06d4aff835d691c5206464f" exitCode=0 Jul 13 21:30:00.406695 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.406646 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" event={"ID":"427b4e67420246f4f9a4a3e967ae607f","Type":"ContainerDied","Data":"7ee586a9aaffdd106b89fb6a0a4da0b539cbd277d06d4aff835d691c5206464f"} Jul 13 21:30:00.408151 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.408129 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" event={"ID":"052647b0243d11ac5a4990de0eb190ae","Type":"ContainerStarted","Data":"dc7fd7ecf881a21b9c96f02eb8d5ef422148091b89896b166a23eaecc6f04b09"} Jul 13 21:30:00.409507 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.409486 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dlbfr" event={"ID":"6732ff06-54b6-4879-9977-4bc85ce107da","Type":"ContainerStarted","Data":"60271b216dcba4bdb8ac0f97d03dd906d552a714aa52e36e5be3f29a53ee9136"} Jul 13 21:30:00.463838 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.463763 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-19.ec2.internal" podStartSLOduration=19.46374615 podStartE2EDuration="19.46374615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:00.462664855 +0000 UTC m=+21.282174689" watchObservedRunningTime="2026-07-13 21:30:00.46374615 +0000 UTC m=+21.283255958" Jul 13 21:30:00.463983 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.463958 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qbrv2" podStartSLOduration=2.304873237 podStartE2EDuration="20.463947524s" totalImagesPullingTime="18.159074287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:00.447607676 +0000 UTC m=+21.267117488" watchObservedRunningTime="2026-07-13 21:30:00.463947524 +0000 UTC m=+21.283457332" Jul 13 21:30:00.478232 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.478193 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bd4p9" podStartSLOduration=2.243530575 podStartE2EDuration="20.478175096s" totalImagesPullingTime="18.234644521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:00.477679451 +0000 UTC m=+21.297189260" watchObservedRunningTime="2026-07-13 21:30:00.478175096 +0000 UTC m=+21.297684905" Jul 13 21:30:00.492259 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.492208 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tklwp" podStartSLOduration=2.377837038 podStartE2EDuration="20.492186366s" totalImagesPullingTime="18.114349328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:00.491109084 +0000 UTC m=+21.310618893" watchObservedRunningTime="2026-07-13 21:30:00.492186366 +0000 UTC m=+21.311696186" Jul 13 21:30:00.504537 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.504489 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vhlcp" podStartSLOduration=10.255135582 podStartE2EDuration="20.504470807s" totalImagesPullingTime="10.249335225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:00.504005647 +0000 UTC m=+21.323515454" watchObservedRunningTime="2026-07-13 21:30:00.504470807 +0000 UTC m=+21.323980619" Jul 13 21:30:00.517100 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:00.517055 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dlbfr" podStartSLOduration=10.275299743 podStartE2EDuration="20.517039947s" totalImagesPullingTime="10.241740204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:00.516430035 +0000 UTC m=+21.335939844" watchObservedRunningTime="2026-07-13 21:30:00.517039947 +0000 UTC m=+21.336549786" Jul 13 21:30:01.345861 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.345686 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:01.345946 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.345734 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:01.345946 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.345748 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:01.346029 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:01.345942 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:01.346072 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:01.346054 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:01.346128 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:01.346113 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:01.348812 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.348799 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:30:01.349251 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.349239 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:30:01.412554 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.412529 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" event={"ID":"427b4e67420246f4f9a4a3e967ae607f","Type":"ContainerStarted","Data":"809e7212b14f61f0077832cbc4ca4fc90864711fef2901086269047cb53180b7"} Jul 13 21:30:01.422354 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.422332 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:30:01.422677 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.422657 2583 generic.go:356] "Generic (PLEG): container finished" podID="f6839532-b922-48be-8590-c4dab166cc5a" containerID="ae0f34b185bcad939df8fe17a4816d748e197abb18b40815ad31b70eebdfc75f" exitCode=1 Jul 13 21:30:01.422749 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.422717 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"ec775c95a14ced8151d834419743eecfa5e3b8545a2385cda297c2b6481eeb2e"} Jul 13 21:30:01.422826 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.422795 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"b9608fa85d1b08e1bf0dd15b58fd69f57d157ce8c99d20b19cf64a4e1f075de5"} Jul 13 21:30:01.422876 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.422866 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"7220e6ef7084a99fc3331af639e6f9ef6aaff15f854ec1a61ccdc7d4dd6c1744"} Jul 13 21:30:01.423374 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.423356 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"b1ee47a333b21ab685fc748f71a313503a97a532b7dc336e4daccfc32a960c99"} Jul 13 21:30:01.423455 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.423394 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerDied","Data":"ae0f34b185bcad939df8fe17a4816d748e197abb18b40815ad31b70eebdfc75f"} Jul 13 21:30:01.423455 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.423423 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"05fa85e8879454ab0e2ddb3d4ac0e07ea8158389b5496593bf2d8ed8286acabb"} Jul 13 21:30:01.424150 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.424131 2583 generic.go:356] "Generic (PLEG): container finished" podID="f688fd0b-cb34-4172-8905-92a5f9a081db" containerID="9c023c9d3ab6995cb55a3c3be6c97e2690903fd7dbcd1f24dea81c42ac1189da" exitCode=0 Jul 13 21:30:01.424218 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.424192 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerDied","Data":"9c023c9d3ab6995cb55a3c3be6c97e2690903fd7dbcd1f24dea81c42ac1189da"} Jul 13 21:30:01.428050 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.428001 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-19.ec2.internal" podStartSLOduration=20.427983408 podStartE2EDuration="20.427983408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:01.427091492 +0000 UTC m=+22.246601302" watchObservedRunningTime="2026-07-13 21:30:01.427983408 +0000 UTC m=+22.247493220" Jul 13 21:30:01.428656 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.428635 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pppts" event={"ID":"c18c033c-9b29-4d74-bede-533cac01c90b","Type":"ContainerStarted","Data":"135d71dbbfa80159c0a482a3556826d9cb92741b8a0cec329131637d073a2987"} Jul 13 21:30:01.429875 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.429855 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:30:01.429989 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.429910 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tklwp" Jul 13 21:30:01.460348 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.460308 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pppts" podStartSLOduration=3.371716476 podStartE2EDuration="21.460293486s" totalImagesPullingTime="18.08857701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:01.459624507 +0000 UTC m=+22.279134315" watchObservedRunningTime="2026-07-13 21:30:01.460293486 +0000 UTC m=+22.279803292" Jul 13 21:30:01.462683 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:01.462661 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 21:30:02.204909 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:02.204823 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T21:30:01.462678587Z","UUID":"ef5ac1fe-1d04-4ef4-a819-50880124e60e","Handler":null,"Name":"","Endpoint":""} Jul 13 21:30:02.206139 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:02.206120 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 21:30:02.206261 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:02.206159 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 21:30:02.433116 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:02.433080 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" event={"ID":"d7f570a4-0f69-4f87-9aa1-161c25c793ac","Type":"ContainerStarted","Data":"b1fd836bc56d608fbcc999306049209437a9f6480e591afe4fa4d9ded099e8bc"} Jul 13 21:30:03.346453 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:03.346251 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:03.346583 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:03.346296 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:03.346583 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:03.346484 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:03.346583 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:03.346306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:03.346681 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:03.346562 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:03.346681 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:03.346660 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:03.436707 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:03.436678 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" event={"ID":"d7f570a4-0f69-4f87-9aa1-161c25c793ac","Type":"ContainerStarted","Data":"a6ce06e7226ac0fcfcde166c74f61529b861a3ffb27aa766d4ee25b1140117bf"} Jul 13 21:30:03.453719 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:03.453683 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jqpzq" podStartSLOduration=2.799541476 podStartE2EDuration="23.453670525s" totalImagesPullingTime="20.654129049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:03.453542588 +0000 UTC m=+24.273052397" watchObservedRunningTime="2026-07-13 21:30:03.453670525 +0000 UTC m=+24.273180346" Jul 13 21:30:04.441839 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:04.441807 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:30:04.442418 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:04.442271 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"e3f93e9cf3750e2fc14c9c3cfaa8156a8f8467ed51668b6006a5738c4fecbe1b"} Jul 13 21:30:05.346115 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:05.346084 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:05.346276 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:05.346199 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:05.346276 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:05.346204 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:05.346276 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:05.346234 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:05.346412 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:05.346342 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:05.346447 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:05.346422 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:07.346367 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.346295 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:07.346367 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.346328 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:07.347079 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.346295 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:07.347079 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:07.346445 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:07.347079 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:07.346515 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:07.347079 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:07.346605 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:07.451311 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.451293 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:30:07.451664 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.451642 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"0ad4ce69d9559b910486fd0116b818c427b8a000df185fbce8fb3d1128d1b98b"} Jul 13 21:30:07.451932 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.451905 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:30:07.452108 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.452089 2583 scope.go:117] "RemoveContainer" containerID="ae0f34b185bcad939df8fe17a4816d748e197abb18b40815ad31b70eebdfc75f" Jul 13 21:30:07.453743 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.453717 2583 generic.go:356] "Generic (PLEG): container finished" podID="f688fd0b-cb34-4172-8905-92a5f9a081db" containerID="8e21f27a56f748bc6e587c9e9d13f1dfbc32fc1f71a2472436484c9878493535" exitCode=0 Jul 13 21:30:07.453828 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.453756 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerDied","Data":"8e21f27a56f748bc6e587c9e9d13f1dfbc32fc1f71a2472436484c9878493535"} Jul 13 21:30:07.470590 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:07.470569 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:30:08.462573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.462551 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:30:08.464730 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.463077 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" event={"ID":"f6839532-b922-48be-8590-c4dab166cc5a","Type":"ContainerStarted","Data":"f3649b71061af588306ef897cff15d301d60a500faef2e3167024764b7b29f6c"} Jul 13 21:30:08.464730 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.463579 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:30:08.464730 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.463646 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:30:08.467040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.467013 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerStarted","Data":"697c982574e19b7a5b83f93e3f4c03511fd3ed963993e6a6533c9e44de48e086"} Jul 13 21:30:08.480501 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.480382 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:30:08.492718 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.492670 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" podStartSLOduration=9.853825885 podStartE2EDuration="28.492651037s" totalImagesPullingTime="18.638825152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:08.491931693 +0000 UTC m=+29.311441520" watchObservedRunningTime="2026-07-13 21:30:08.492651037 +0000 UTC m=+29.312160844" Jul 13 21:30:08.677826 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.677756 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-49ngc"] Jul 13 21:30:08.678001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.677890 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:08.678069 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:08.678000 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:08.680809 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.680785 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ckzb9"] Jul 13 21:30:08.680924 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.680871 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:08.680977 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:08.680952 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:08.681389 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.681366 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcl7j"] Jul 13 21:30:08.681478 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:08.681465 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:08.681567 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:08.681550 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:09.471792 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:09.471728 2583 generic.go:356] "Generic (PLEG): container finished" podID="f688fd0b-cb34-4172-8905-92a5f9a081db" containerID="697c982574e19b7a5b83f93e3f4c03511fd3ed963993e6a6533c9e44de48e086" exitCode=0 Jul 13 21:30:09.472218 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:09.471802 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerDied","Data":"697c982574e19b7a5b83f93e3f4c03511fd3ed963993e6a6533c9e44de48e086"} Jul 13 21:30:10.347462 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:10.347432 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:10.347575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:10.347518 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:10.347575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:10.347550 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:10.347659 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:10.347587 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:10.347659 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:10.347636 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:10.347738 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:10.347718 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:11.479324 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:11.479293 2583 generic.go:356] "Generic (PLEG): container finished" podID="f688fd0b-cb34-4172-8905-92a5f9a081db" containerID="c2af77ce1e9db17370b45652c6fc57eadea2c9a6b699884a24a55e65ff49881c" exitCode=0 Jul 13 21:30:11.479923 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:11.479345 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerDied","Data":"c2af77ce1e9db17370b45652c6fc57eadea2c9a6b699884a24a55e65ff49881c"} Jul 13 21:30:12.345835 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:12.345787 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:12.345835 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:12.345808 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:12.346068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:12.345787 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:12.346068 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:12.345942 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:30:12.346068 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:12.346055 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckzb9" podUID="4f589797-94ec-4487-b525-157b6547a3d5" Jul 13 21:30:12.346234 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:12.346117 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcl7j" podUID="3741693b-7718-4bd8-9d6d-4885316b0b89" Jul 13 21:30:13.397992 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:13.397940 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:13.398458 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:13.398035 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:13.398458 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.398096 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:30:13.398458 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.398138 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:30:13.398458 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.398153 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:45.398139447 +0000 UTC m=+66.217649237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:30:13.398458 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.398188 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret podName:4f589797-94ec-4487-b525-157b6547a3d5 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:45.398173157 +0000 UTC m=+66.217682947 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret") pod "global-pull-secret-syncer-ckzb9" (UID: "4f589797-94ec-4487-b525-157b6547a3d5") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:30:13.498689 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:13.498664 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:13.498865 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.498851 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:30:13.498918 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.498871 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:30:13.498918 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.498883 2583 projected.go:196] Error preparing data for projected volume kube-api-access-m5k9f for pod openshift-network-diagnostics/network-check-target-bcl7j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:30:13.498996 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:13.498934 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f podName:3741693b-7718-4bd8-9d6d-4885316b0b89 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:45.498915514 +0000 UTC m=+66.318425305 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5k9f" (UniqueName: "kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f") pod "network-check-target-bcl7j" (UID: "3741693b-7718-4bd8-9d6d-4885316b0b89") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:30:13.930497 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:13.930439 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-19.ec2.internal" event="NodeReady" Jul 13 21:30:13.930731 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:13.930546 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 21:30:13.979071 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:13.979033 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4ddkh"] Jul 13 21:30:14.008951 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.008923 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fjmwk"] Jul 13 21:30:14.009090 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.009044 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.011343 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.011315 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 21:30:14.011343 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.011334 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l6v89\"" Jul 13 21:30:14.011514 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.011364 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 21:30:14.026953 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.026935 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fjmwk"] Jul 13 21:30:14.027042 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.026971 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4ddkh"] Jul 13 21:30:14.027094 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.027042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.029540 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.029521 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wlljv\"" Jul 13 21:30:14.029637 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.029552 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 21:30:14.029637 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.029601 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 21:30:14.029838 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.029808 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 21:30:14.205525 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.205440 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5192b507-e857-4d28-bd49-ec838b6d1664-cert\") pod \"ingress-canary-fjmwk\" (UID: \"5192b507-e857-4d28-bd49-ec838b6d1664\") " pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.205525 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.205486 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh2jt\" (UniqueName: \"kubernetes.io/projected/5192b507-e857-4d28-bd49-ec838b6d1664-kube-api-access-sh2jt\") pod \"ingress-canary-fjmwk\" (UID: \"5192b507-e857-4d28-bd49-ec838b6d1664\") " pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.205525 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.205514 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v49fp\" (UniqueName: \"kubernetes.io/projected/de0238bd-168e-48aa-9f7c-d8cdea513774-kube-api-access-v49fp\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.205830 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.205597 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de0238bd-168e-48aa-9f7c-d8cdea513774-config-volume\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.205830 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.205654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0238bd-168e-48aa-9f7c-d8cdea513774-tmp-dir\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.205830 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.205697 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.306367 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306340 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5192b507-e857-4d28-bd49-ec838b6d1664-cert\") pod \"ingress-canary-fjmwk\" (UID: \"5192b507-e857-4d28-bd49-ec838b6d1664\") " pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.306548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh2jt\" (UniqueName: \"kubernetes.io/projected/5192b507-e857-4d28-bd49-ec838b6d1664-kube-api-access-sh2jt\") pod \"ingress-canary-fjmwk\" (UID: \"5192b507-e857-4d28-bd49-ec838b6d1664\") " pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.306548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v49fp\" (UniqueName: \"kubernetes.io/projected/de0238bd-168e-48aa-9f7c-d8cdea513774-kube-api-access-v49fp\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.306548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de0238bd-168e-48aa-9f7c-d8cdea513774-config-volume\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.306548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306473 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0238bd-168e-48aa-9f7c-d8cdea513774-tmp-dir\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.306548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.306810 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:14.306607 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:14.306810 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:14.306672 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:14.806652357 +0000 UTC m=+35.626162160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:14.306925 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.306880 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0238bd-168e-48aa-9f7c-d8cdea513774-tmp-dir\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.307076 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.307053 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de0238bd-168e-48aa-9f7c-d8cdea513774-config-volume\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.311040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.310899 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5192b507-e857-4d28-bd49-ec838b6d1664-cert\") pod \"ingress-canary-fjmwk\" (UID: \"5192b507-e857-4d28-bd49-ec838b6d1664\") " pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.315546 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.315522 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh2jt\" (UniqueName: \"kubernetes.io/projected/5192b507-e857-4d28-bd49-ec838b6d1664-kube-api-access-sh2jt\") pod \"ingress-canary-fjmwk\" (UID: \"5192b507-e857-4d28-bd49-ec838b6d1664\") " pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.315895 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.315871 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v49fp\" (UniqueName: \"kubernetes.io/projected/de0238bd-168e-48aa-9f7c-d8cdea513774-kube-api-access-v49fp\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.336862 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.336835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fjmwk" Jul 13 21:30:14.346160 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.346134 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:14.346259 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.346245 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:14.346319 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.346276 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:14.350037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.349767 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 21:30:14.350037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.349814 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 21:30:14.350037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.349839 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89gzj\"" Jul 13 21:30:14.350037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.349860 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 21:30:14.350037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.349814 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qsrtw\"" Jul 13 21:30:14.350037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.349791 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 21:30:14.811279 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:14.811245 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:14.811922 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:14.811376 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:14.811922 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:14.811467 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:15.811445469 +0000 UTC m=+36.630955267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:15.622326 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:15.622301 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fjmwk"] Jul 13 21:30:15.685921 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:30:15.685883 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5192b507_e857_4d28_bd49_ec838b6d1664.slice/crio-0a7234f1d2bf1dc91be9e31110c4ae40189e0633942e6378aa83ea0a2726e11c WatchSource:0}: Error finding container 0a7234f1d2bf1dc91be9e31110c4ae40189e0633942e6378aa83ea0a2726e11c: Status 404 returned error can't find the container with id 0a7234f1d2bf1dc91be9e31110c4ae40189e0633942e6378aa83ea0a2726e11c Jul 13 21:30:15.819837 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:15.819805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:15.820128 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:15.819950 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:15.820128 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:15.820001 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:17.819985474 +0000 UTC m=+38.639495266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:16.491008 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:16.490973 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fjmwk" event={"ID":"5192b507-e857-4d28-bd49-ec838b6d1664","Type":"ContainerStarted","Data":"0a7234f1d2bf1dc91be9e31110c4ae40189e0633942e6378aa83ea0a2726e11c"} Jul 13 21:30:16.495001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:16.494970 2583 generic.go:356] "Generic (PLEG): container finished" podID="f688fd0b-cb34-4172-8905-92a5f9a081db" containerID="b44b0f97e20f0113d6c677830fde61141617c71eae59d86b67f4452b6d076c6c" exitCode=0 Jul 13 21:30:16.495001 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:16.495010 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerDied","Data":"b44b0f97e20f0113d6c677830fde61141617c71eae59d86b67f4452b6d076c6c"} Jul 13 21:30:17.500820 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:17.500632 2583 generic.go:356] "Generic (PLEG): container finished" podID="f688fd0b-cb34-4172-8905-92a5f9a081db" containerID="878400a10c07ccc677b01a2d672dc0e76501b695a6b4e6202d25922be05ef37e" exitCode=0 Jul 13 21:30:17.501184 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:17.500721 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerDied","Data":"878400a10c07ccc677b01a2d672dc0e76501b695a6b4e6202d25922be05ef37e"} Jul 13 21:30:17.835877 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:17.835789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:17.836020 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:17.835944 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:17.836020 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:17.836017 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:21.835994413 +0000 UTC m=+42.655504200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:18.506499 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:18.506436 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" event={"ID":"f688fd0b-cb34-4172-8905-92a5f9a081db","Type":"ContainerStarted","Data":"cb67cb0ec76aea72e08685593d8f49b1467f67a757b7fc751c440dbee3c26268"} Jul 13 21:30:18.531035 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:18.530989 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gb9zp" podStartSLOduration=8.804698584 podStartE2EDuration="38.530975819s" totalImagesPullingTime="29.726277235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:18.529761419 +0000 UTC m=+39.349271226" watchObservedRunningTime="2026-07-13 21:30:18.530975819 +0000 UTC m=+39.350485627" Jul 13 21:30:19.509811 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:19.509779 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fjmwk" event={"ID":"5192b507-e857-4d28-bd49-ec838b6d1664","Type":"ContainerStarted","Data":"cf567dbb39b4f7efa229a9bec321e8adc668f35924740a42ba92dbfc527ab863"} Jul 13 21:30:19.525394 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:19.525354 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fjmwk" podStartSLOduration=3.678868967 podStartE2EDuration="6.52534011s" totalImagesPullingTime="2.846471143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:19.525263689 +0000 UTC m=+40.344773515" watchObservedRunningTime="2026-07-13 21:30:19.52534011 +0000 UTC m=+40.344849918" Jul 13 21:30:21.861629 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:21.861590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:21.862129 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:21.861713 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:21.862129 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:21.861804 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:29.861762028 +0000 UTC m=+50.681271817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:29.912098 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:29.912064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:29.912467 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:29.912206 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:29.912467 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:29.912269 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:45.912253454 +0000 UTC m=+66.731763245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:40.453596 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.453568 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd"] Jul 13 21:30:40.484581 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.484555 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw"] Jul 13 21:30:40.484703 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.484667 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.487415 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.487396 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 21:30:40.487789 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.487752 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 21:30:40.488512 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.488494 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 13 21:30:40.488512 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.488508 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 21:30:40.488626 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.488535 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xtbqj\"" Jul 13 21:30:40.496179 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.496161 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb7hh" Jul 13 21:30:40.496265 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.496219 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.496265 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.496231 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd"] Jul 13 21:30:40.496265 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.496253 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw"] Jul 13 21:30:40.498671 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.498654 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 13 21:30:40.498762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.498730 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 13 21:30:40.498762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.498755 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 13 21:30:40.498944 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.498929 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 13 21:30:40.574465 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.574554 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574471 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68lv8\" (UniqueName: \"kubernetes.io/projected/325e76f5-23f6-4be3-980a-c407486f81f8-kube-api-access-68lv8\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.574554 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574506 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xppnw\" (UniqueName: \"kubernetes.io/projected/27b2dc03-5b4f-44bd-a6aa-c249797e01ea-kube-api-access-xppnw\") pod \"managed-serviceaccount-addon-agent-78f47786d4-zbtmd\" (UID: \"27b2dc03-5b4f-44bd-a6aa-c249797e01ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.574697 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574556 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/325e76f5-23f6-4be3-980a-c407486f81f8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.574697 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574573 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.574796 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/27b2dc03-5b4f-44bd-a6aa-c249797e01ea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78f47786d4-zbtmd\" (UID: \"27b2dc03-5b4f-44bd-a6aa-c249797e01ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.574847 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574836 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-ca\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.574882 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.574852 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-hub\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.675874 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.675844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/27b2dc03-5b4f-44bd-a6aa-c249797e01ea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78f47786d4-zbtmd\" (UID: \"27b2dc03-5b4f-44bd-a6aa-c249797e01ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.675967 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.675884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-ca\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.675967 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.675900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-hub\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.675967 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.675915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.675967 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.675930 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68lv8\" (UniqueName: \"kubernetes.io/projected/325e76f5-23f6-4be3-980a-c407486f81f8-kube-api-access-68lv8\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.676166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.676046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xppnw\" (UniqueName: \"kubernetes.io/projected/27b2dc03-5b4f-44bd-a6aa-c249797e01ea-kube-api-access-xppnw\") pod \"managed-serviceaccount-addon-agent-78f47786d4-zbtmd\" (UID: \"27b2dc03-5b4f-44bd-a6aa-c249797e01ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.676166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.676106 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/325e76f5-23f6-4be3-980a-c407486f81f8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.676166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.676140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.676729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.676709 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/325e76f5-23f6-4be3-980a-c407486f81f8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.679825 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.679799 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/27b2dc03-5b4f-44bd-a6aa-c249797e01ea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78f47786d4-zbtmd\" (UID: \"27b2dc03-5b4f-44bd-a6aa-c249797e01ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.679930 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.679850 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-ca\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.679930 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.679883 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.680049 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.679934 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-hub\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.680121 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.680101 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/325e76f5-23f6-4be3-980a-c407486f81f8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.683561 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.683540 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68lv8\" (UniqueName: \"kubernetes.io/projected/325e76f5-23f6-4be3-980a-c407486f81f8-kube-api-access-68lv8\") pod \"cluster-proxy-proxy-agent-646668dfff-2hvlw\" (UID: \"325e76f5-23f6-4be3-980a-c407486f81f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.683666 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.683647 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xppnw\" (UniqueName: \"kubernetes.io/projected/27b2dc03-5b4f-44bd-a6aa-c249797e01ea-kube-api-access-xppnw\") pod \"managed-serviceaccount-addon-agent-78f47786d4-zbtmd\" (UID: \"27b2dc03-5b4f-44bd-a6aa-c249797e01ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.803354 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.803294 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" Jul 13 21:30:40.809489 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.809471 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:30:40.925749 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.925715 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd"] Jul 13 21:30:40.929230 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:30:40.929205 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27b2dc03_5b4f_44bd_a6aa_c249797e01ea.slice/crio-7436376d89a1d224622689906de42134de2357e34d0ab5e3b7623bde024e9b86 WatchSource:0}: Error finding container 7436376d89a1d224622689906de42134de2357e34d0ab5e3b7623bde024e9b86: Status 404 returned error can't find the container with id 7436376d89a1d224622689906de42134de2357e34d0ab5e3b7623bde024e9b86 Jul 13 21:30:40.943081 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:40.943058 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw"] Jul 13 21:30:40.945872 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:30:40.945848 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod325e76f5_23f6_4be3_980a_c407486f81f8.slice/crio-7e317201f3873f8fc3bec4acb9bde5774f878bdcda7bb08f3c27bad85e5a1a76 WatchSource:0}: Error finding container 7e317201f3873f8fc3bec4acb9bde5774f878bdcda7bb08f3c27bad85e5a1a76: Status 404 returned error can't find the container with id 7e317201f3873f8fc3bec4acb9bde5774f878bdcda7bb08f3c27bad85e5a1a76 Jul 13 21:30:41.557830 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:41.557793 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" event={"ID":"325e76f5-23f6-4be3-980a-c407486f81f8","Type":"ContainerStarted","Data":"7e317201f3873f8fc3bec4acb9bde5774f878bdcda7bb08f3c27bad85e5a1a76"} Jul 13 21:30:41.558870 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:41.558851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" event={"ID":"27b2dc03-5b4f-44bd-a6aa-c249797e01ea","Type":"ContainerStarted","Data":"7436376d89a1d224622689906de42134de2357e34d0ab5e3b7623bde024e9b86"} Jul 13 21:30:45.415139 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.415092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:30:45.415563 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.415164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:45.418266 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.418012 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 21:30:45.418266 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.418144 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 21:30:45.425380 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:45.425356 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 21:30:45.425489 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:45.425428 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:31:49.425408205 +0000 UTC m=+130.244918005 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : secret "metrics-daemon-secret" not found Jul 13 21:30:45.427490 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.427464 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f589797-94ec-4487-b525-157b6547a3d5-original-pull-secret\") pod \"global-pull-secret-syncer-ckzb9\" (UID: \"4f589797-94ec-4487-b525-157b6547a3d5\") " pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:45.516448 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.516411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:45.519080 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.519061 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 21:30:45.529321 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.529301 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 21:30:45.539539 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.539518 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5k9f\" (UniqueName: \"kubernetes.io/projected/3741693b-7718-4bd8-9d6d-4885316b0b89-kube-api-access-m5k9f\") pod \"network-check-target-bcl7j\" (UID: \"3741693b-7718-4bd8-9d6d-4885316b0b89\") " pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:45.558333 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.558303 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckzb9" Jul 13 21:30:45.571994 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.571969 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qsrtw\"" Jul 13 21:30:45.572193 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.572166 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" event={"ID":"325e76f5-23f6-4be3-980a-c407486f81f8","Type":"ContainerStarted","Data":"c09f7facf77c8c7b4005c4e723c251a1632999cf4dbf4f3800d3cc5d185099ee"} Jul 13 21:30:45.572282 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.572210 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" event={"ID":"325e76f5-23f6-4be3-980a-c407486f81f8","Type":"ContainerStarted","Data":"1dea6b423f9bda6c1cde2dfd1bedca2167c645e34e0ca13c798627630c511228"} Jul 13 21:30:45.572282 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.572228 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" event={"ID":"325e76f5-23f6-4be3-980a-c407486f81f8","Type":"ContainerStarted","Data":"5fe0d0b5ef2aef5304a1df60b2127813e0a457dee1905c4578a70ea15a522f07"} Jul 13 21:30:45.573514 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.573494 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" event={"ID":"27b2dc03-5b4f-44bd-a6aa-c249797e01ea","Type":"ContainerStarted","Data":"044865356e41ba9414fbba3688e3ff51a029b5c97a42e10a134f2bd5bd9595a9"} Jul 13 21:30:45.580392 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.580375 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:45.591654 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.591604 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" podStartSLOduration=1.712380864 podStartE2EDuration="5.591584364s" totalImagesPullingTime="3.8792035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:45.590275385 +0000 UTC m=+66.409785195" watchObservedRunningTime="2026-07-13 21:30:45.591584364 +0000 UTC m=+66.411094175" Jul 13 21:30:45.604339 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.604289 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" podStartSLOduration=1.722715006 podStartE2EDuration="5.60427527s" totalImagesPullingTime="3.881560264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:45.603878924 +0000 UTC m=+66.423388737" watchObservedRunningTime="2026-07-13 21:30:45.60427527 +0000 UTC m=+66.423785079" Jul 13 21:30:45.685450 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.685418 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ckzb9"] Jul 13 21:30:45.689296 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:30:45.689270 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f589797_94ec_4487_b525_157b6547a3d5.slice/crio-f42952b7a87bd2f5c85393f43345b235b33e20c8732f5b95081f5781fefc0a64 WatchSource:0}: Error finding container f42952b7a87bd2f5c85393f43345b235b33e20c8732f5b95081f5781fefc0a64: Status 404 returned error can't find the container with id f42952b7a87bd2f5c85393f43345b235b33e20c8732f5b95081f5781fefc0a64 Jul 13 21:30:45.710801 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.710764 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcl7j"] Jul 13 21:30:45.713487 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:30:45.713463 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3741693b_7718_4bd8_9d6d_4885316b0b89.slice/crio-d6f03ff447fae0b0740ec0f5974cdb24433acfb7a7188e68d21f77f477a56668 WatchSource:0}: Error finding container d6f03ff447fae0b0740ec0f5974cdb24433acfb7a7188e68d21f77f477a56668: Status 404 returned error can't find the container with id d6f03ff447fae0b0740ec0f5974cdb24433acfb7a7188e68d21f77f477a56668 Jul 13 21:30:45.919926 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:45.919845 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:30:45.920062 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:45.919949 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:30:45.920062 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:30:45.920004 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:31:17.919990165 +0000 UTC m=+98.739499952 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:30:46.578469 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:46.578425 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcl7j" event={"ID":"3741693b-7718-4bd8-9d6d-4885316b0b89","Type":"ContainerStarted","Data":"d6f03ff447fae0b0740ec0f5974cdb24433acfb7a7188e68d21f77f477a56668"} Jul 13 21:30:46.580303 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:46.580261 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ckzb9" event={"ID":"4f589797-94ec-4487-b525-157b6547a3d5","Type":"ContainerStarted","Data":"f42952b7a87bd2f5c85393f43345b235b33e20c8732f5b95081f5781fefc0a64"} Jul 13 21:30:49.590087 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:49.590056 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcl7j" event={"ID":"3741693b-7718-4bd8-9d6d-4885316b0b89","Type":"ContainerStarted","Data":"803ebd862006e2f633ff603babf872e0f3190971419e843b78efc37a0b0e9c54"} Jul 13 21:30:49.590434 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:49.590183 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:30:49.606391 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:49.606345 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bcl7j" podStartSLOduration=66.31513734 podStartE2EDuration="1m9.60633055s" totalImagesPullingTime="3.29119321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:49.604817438 +0000 UTC m=+70.424327270" watchObservedRunningTime="2026-07-13 21:30:49.60633055 +0000 UTC m=+70.425840358" Jul 13 21:30:51.597637 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:51.597603 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ckzb9" event={"ID":"4f589797-94ec-4487-b525-157b6547a3d5","Type":"ContainerStarted","Data":"37967ab81812504195b2bf010188ccd3d37d8edf0a02d95007310413a23abc7f"} Jul 13 21:30:51.611399 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:30:51.611284 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ckzb9" podStartSLOduration=65.877805787 podStartE2EDuration="1m11.611267475s" totalImagesPullingTime="5.733461688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:51.610956169 +0000 UTC m=+72.430465978" watchObservedRunningTime="2026-07-13 21:30:51.611267475 +0000 UTC m=+72.430777287" Jul 13 21:31:17.926172 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:31:17.926138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:31:17.926643 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:31:17.926254 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:31:17.926643 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:31:17.926306 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:32:21.926291456 +0000 UTC m=+162.745801242 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:31:20.595401 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:31:20.595375 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bcl7j" Jul 13 21:31:49.435583 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:31:49.435541 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:31:49.436131 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:31:49.435666 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 21:31:49.436131 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:31:49.435754 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:51.435736564 +0000 UTC m=+252.255246351 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : secret "metrics-daemon-secret" not found Jul 13 21:32:17.020088 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:32:17.020048 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4ddkh" podUID="de0238bd-168e-48aa-9f7c-d8cdea513774" Jul 13 21:32:17.364123 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:32:17.364037 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:32:17.843198 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:17.843168 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4ddkh" Jul 13 21:32:21.940371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:21.940333 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:32:21.940816 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:32:21.940454 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 21:32:21.940816 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:32:21.940506 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls podName:de0238bd-168e-48aa-9f7c-d8cdea513774 nodeName:}" failed. No retries permitted until 2026-07-13 21:34:23.940492199 +0000 UTC m=+284.760001987 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls") pod "dns-default-4ddkh" (UID: "de0238bd-168e-48aa-9f7c-d8cdea513774") : secret "dns-default-metrics-tls" not found Jul 13 21:32:32.346054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:32.345964 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:32:45.928114 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:45.928075 2583 generic.go:356] "Generic (PLEG): container finished" podID="27b2dc03-5b4f-44bd-a6aa-c249797e01ea" containerID="044865356e41ba9414fbba3688e3ff51a029b5c97a42e10a134f2bd5bd9595a9" exitCode=255 Jul 13 21:32:45.928510 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:45.928149 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" event={"ID":"27b2dc03-5b4f-44bd-a6aa-c249797e01ea","Type":"ContainerDied","Data":"044865356e41ba9414fbba3688e3ff51a029b5c97a42e10a134f2bd5bd9595a9"} Jul 13 21:32:45.928510 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:45.928431 2583 scope.go:117] "RemoveContainer" containerID="044865356e41ba9414fbba3688e3ff51a029b5c97a42e10a134f2bd5bd9595a9" Jul 13 21:32:46.933145 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:32:46.933105 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78f47786d4-zbtmd" event={"ID":"27b2dc03-5b4f-44bd-a6aa-c249797e01ea","Type":"ContainerStarted","Data":"7e9c2f80abc2ae00ba1434dfc3c3ead62fcc224e21b18d775acd1e607dcd0421"} Jul 13 21:33:43.802557 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.802524 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vm95p"] Jul 13 21:33:43.805280 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.805263 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.807994 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.807968 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 21:33:43.808120 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.807968 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 21:33:43.808432 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.808416 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 21:33:43.808523 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.808507 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tf6cc\"" Jul 13 21:33:43.808954 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.808940 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 21:33:43.816929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.816910 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vm95p"] Jul 13 21:33:43.838250 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.838228 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-crio-socket\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.838338 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.838252 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.838338 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.838273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.838338 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.838289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w77hm\" (UniqueName: \"kubernetes.io/projected/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-api-access-w77hm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.838338 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.838319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-data-volume\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939148 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939122 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-crio-socket\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939244 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939154 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939244 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939346 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939239 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-crio-socket\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939346 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:43.939256 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 21:33:43.939346 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:43.939277 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 21:33:43.939346 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77hm\" (UniqueName: \"kubernetes.io/projected/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-api-access-w77hm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939346 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:43.939321 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:44.439302631 +0000 UTC m=+245.258812438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : configmap "kube-rbac-proxy" not found Jul 13 21:33:43.939531 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:43.939366 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:44.439347538 +0000 UTC m=+245.258857330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : secret "insights-runtime-extractor-tls" not found Jul 13 21:33:43.939531 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-data-volume\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.939680 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.939664 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-data-volume\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:43.947691 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:43.947673 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w77hm\" (UniqueName: \"kubernetes.io/projected/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-api-access-w77hm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:44.442861 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:44.442820 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:44.442861 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:44.442867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:44.443087 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:44.442970 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 21:33:44.443087 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:44.442992 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 21:33:44.443087 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:44.443060 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:45.443041152 +0000 UTC m=+246.262550939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : secret "insights-runtime-extractor-tls" not found Jul 13 21:33:44.443087 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:44.443081 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:45.443071883 +0000 UTC m=+246.262581698 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : configmap "kube-rbac-proxy" not found Jul 13 21:33:45.450182 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:45.450148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:45.450602 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:45.450342 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 21:33:45.450602 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:45.450398 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:47.450382718 +0000 UTC m=+248.269892506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : secret "insights-runtime-extractor-tls" not found Jul 13 21:33:45.450602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:45.450505 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:45.450602 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:45.450589 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 21:33:45.450799 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:45.450657 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:47.450639524 +0000 UTC m=+248.270149317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : configmap "kube-rbac-proxy" not found Jul 13 21:33:46.788695 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:46.788665 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhlcp_b0438b82-c3d6-4ce8-8d45-fb3ac41c659c/dns-node-resolver/0.log" Jul 13 21:33:47.462302 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:47.462267 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:47.462473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:47.462320 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:47.462473 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:47.462382 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 21:33:47.462473 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:47.462459 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:51.462439923 +0000 UTC m=+252.281949725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : configmap "kube-rbac-proxy" not found Jul 13 21:33:47.462473 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:47.462467 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 21:33:47.462641 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:47.462556 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:51.462543326 +0000 UTC m=+252.282053120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : secret "insights-runtime-extractor-tls" not found Jul 13 21:33:47.587405 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:47.587381 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dlbfr_6732ff06-54b6-4879-9977-4bc85ce107da/node-ca/0.log" Jul 13 21:33:48.387963 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:48.387911 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fjmwk_5192b507-e857-4d28-bd49-ec838b6d1664/serve-healthcheck-canary/0.log" Jul 13 21:33:51.489974 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:51.489940 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:51.489988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:51.490029 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:51.490058 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:51.490103 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:51.490109 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:59.490095677 +0000 UTC m=+260.309605465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : configmap "kube-rbac-proxy" not found Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:51.490142 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs podName:5afe6508-8b7b-4083-ac75-e95e48666e82 nodeName:}" failed. No retries permitted until 2026-07-13 21:35:53.490130395 +0000 UTC m=+374.309640183 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs") pod "network-metrics-daemon-49ngc" (UID: "5afe6508-8b7b-4083-ac75-e95e48666e82") : secret "metrics-daemon-secret" not found Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:51.490153 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 21:33:51.490397 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:33:51.490213 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls podName:6b8c7d68-2b91-48fd-9a1a-403884ff3f53 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:59.49019747 +0000 UTC m=+260.309707272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vm95p" (UID: "6b8c7d68-2b91-48fd-9a1a-403884ff3f53") : secret "insights-runtime-extractor-tls" not found Jul 13 21:33:59.544767 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:59.544732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:59.544767 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:59.544767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:59.545378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:59.545356 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:59.547066 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:59.547045 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6b8c7d68-2b91-48fd-9a1a-403884ff3f53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vm95p\" (UID: \"6b8c7d68-2b91-48fd-9a1a-403884ff3f53\") " pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:59.714458 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:59.714423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vm95p" Jul 13 21:33:59.834952 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:33:59.834889 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vm95p"] Jul 13 21:33:59.838151 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:33:59.838127 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8c7d68_2b91_48fd_9a1a_403884ff3f53.slice/crio-7a22b3130bf3230a25b6591b6a0217ba45556143a61a37c08633388d95c80509 WatchSource:0}: Error finding container 7a22b3130bf3230a25b6591b6a0217ba45556143a61a37c08633388d95c80509: Status 404 returned error can't find the container with id 7a22b3130bf3230a25b6591b6a0217ba45556143a61a37c08633388d95c80509 Jul 13 21:34:00.138297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:00.138220 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vm95p" event={"ID":"6b8c7d68-2b91-48fd-9a1a-403884ff3f53","Type":"ContainerStarted","Data":"7a22b3130bf3230a25b6591b6a0217ba45556143a61a37c08633388d95c80509"} Jul 13 21:34:04.151097 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:04.151068 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vm95p" event={"ID":"6b8c7d68-2b91-48fd-9a1a-403884ff3f53","Type":"ContainerStarted","Data":"c0066325fc062e332be48b215e2109fa883e960df068e8e08a30133812e60110"} Jul 13 21:34:05.156864 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:05.156826 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vm95p" event={"ID":"6b8c7d68-2b91-48fd-9a1a-403884ff3f53","Type":"ContainerStarted","Data":"d84efba43c6d9371b60daf71d2c7b35f35606d7f8d3e43b0f2e5c4fc9c931a1c"} Jul 13 21:34:07.164721 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:07.164688 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vm95p" event={"ID":"6b8c7d68-2b91-48fd-9a1a-403884ff3f53","Type":"ContainerStarted","Data":"c1782cd63bb223cff7fb737d98eb975bec6166fb404efaecb729106a9624771d"} Jul 13 21:34:07.180404 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:07.180364 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vm95p" podStartSLOduration=17.588018506 podStartE2EDuration="24.180349381s" totalImagesPullingTime="6.592330875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:33:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:07.179898443 +0000 UTC m=+267.999408285" watchObservedRunningTime="2026-07-13 21:34:07.180349381 +0000 UTC m=+267.999859190" Jul 13 21:34:20.844104 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:34:20.844069 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4ddkh" podUID="de0238bd-168e-48aa-9f7c-d8cdea513774" Jul 13 21:34:21.206164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.206136 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4ddkh" Jul 13 21:34:21.952099 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.952072 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-7l94x"] Jul 13 21:34:21.958109 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.958089 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:21.960970 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.960946 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 21:34:21.961165 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.961153 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4k24c\"" Jul 13 21:34:21.961277 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.961261 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 21:34:21.973538 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:21.973514 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-7l94x"] Jul 13 21:34:22.100350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:22.100330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpmhd\" (UniqueName: \"kubernetes.io/projected/57d45916-3720-410a-a967-c448673c7233-kube-api-access-jpmhd\") pod \"downloads-68c9cbb497-7l94x\" (UID: \"57d45916-3720-410a-a967-c448673c7233\") " pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:22.200912 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:22.200883 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpmhd\" (UniqueName: \"kubernetes.io/projected/57d45916-3720-410a-a967-c448673c7233-kube-api-access-jpmhd\") pod \"downloads-68c9cbb497-7l94x\" (UID: \"57d45916-3720-410a-a967-c448673c7233\") " pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:22.208898 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:22.208841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpmhd\" (UniqueName: \"kubernetes.io/projected/57d45916-3720-410a-a967-c448673c7233-kube-api-access-jpmhd\") pod \"downloads-68c9cbb497-7l94x\" (UID: \"57d45916-3720-410a-a967-c448673c7233\") " pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:22.266575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:22.266554 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:22.385485 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:22.385452 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-7l94x"] Jul 13 21:34:22.388076 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:34:22.388052 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d45916_3720_410a_a967_c448673c7233.slice/crio-9c7d3dbbda784da7005e3e60517b03539316216c53c0890d49135cbd616eda68 WatchSource:0}: Error finding container 9c7d3dbbda784da7005e3e60517b03539316216c53c0890d49135cbd616eda68: Status 404 returned error can't find the container with id 9c7d3dbbda784da7005e3e60517b03539316216c53c0890d49135cbd616eda68 Jul 13 21:34:23.212262 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.212227 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-7l94x" event={"ID":"57d45916-3720-410a-a967-c448673c7233","Type":"ContainerStarted","Data":"9c7d3dbbda784da7005e3e60517b03539316216c53c0890d49135cbd616eda68"} Jul 13 21:34:23.608923 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.608848 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj"] Jul 13 21:34:23.612060 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.612032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.614642 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.614620 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 21:34:23.614850 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.614830 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 21:34:23.614963 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.614856 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 21:34:23.615928 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.615908 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 21:34:23.616054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.615973 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 21:34:23.616054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.616015 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wndqs\"" Jul 13 21:34:23.625842 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.625821 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj"] Jul 13 21:34:23.634593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.634571 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rv6jn"] Jul 13 21:34:23.637927 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.637905 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.640762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.640726 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vvh8d\"" Jul 13 21:34:23.640881 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.640795 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 21:34:23.640881 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.640842 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 21:34:23.640991 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.640958 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 21:34:23.707527 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae395e51-c106-4727-9615-226822487945-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.707659 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-root\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707659 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707568 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707659 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707627 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-sys\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707795 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707678 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-textfile\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707795 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707707 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-tls\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707795 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707734 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae395e51-c106-4727-9615-226822487945-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.707795 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46chz\" (UniqueName: \"kubernetes.io/projected/3124177d-dc55-454c-b9c9-7a2f3557d4f5-kube-api-access-46chz\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707956 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707825 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707956 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707875 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-wtmp\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.707956 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707919 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3124177d-dc55-454c-b9c9-7a2f3557d4f5-metrics-client-ca\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.708106 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707958 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae395e51-c106-4727-9615-226822487945-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.708106 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.707984 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8br7\" (UniqueName: \"kubernetes.io/projected/ae395e51-c106-4727-9615-226822487945-kube-api-access-t8br7\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.808996 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.808969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-root\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809018 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-sys\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-textfile\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-tls\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809078 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-root\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809095 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae395e51-c106-4727-9615-226822487945-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809114 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-sys\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809132 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46chz\" (UniqueName: \"kubernetes.io/projected/3124177d-dc55-454c-b9c9-7a2f3557d4f5-kube-api-access-46chz\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809196 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-wtmp\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809227 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3124177d-dc55-454c-b9c9-7a2f3557d4f5-metrics-client-ca\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae395e51-c106-4727-9615-226822487945-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809372 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-wtmp\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809394 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-textfile\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8br7\" (UniqueName: \"kubernetes.io/projected/ae395e51-c106-4727-9615-226822487945-kube-api-access-t8br7\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.809602 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.809454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae395e51-c106-4727-9615-226822487945-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.810054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.810008 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae395e51-c106-4727-9615-226822487945-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.810161 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.810054 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3124177d-dc55-454c-b9c9-7a2f3557d4f5-metrics-client-ca\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.810221 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.810170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.812250 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.812211 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae395e51-c106-4727-9615-226822487945-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.812250 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.812231 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae395e51-c106-4727-9615-226822487945-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.812384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.812364 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-tls\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.812384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.812375 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3124177d-dc55-454c-b9c9-7a2f3557d4f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.820713 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.820690 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46chz\" (UniqueName: \"kubernetes.io/projected/3124177d-dc55-454c-b9c9-7a2f3557d4f5-kube-api-access-46chz\") pod \"node-exporter-rv6jn\" (UID: \"3124177d-dc55-454c-b9c9-7a2f3557d4f5\") " pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.820840 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.820713 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8br7\" (UniqueName: \"kubernetes.io/projected/ae395e51-c106-4727-9615-226822487945-kube-api-access-t8br7\") pod \"openshift-state-metrics-84bd68b896-fwhsj\" (UID: \"ae395e51-c106-4727-9615-226822487945\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.923056 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.923034 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" Jul 13 21:34:23.948125 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:23.948017 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rv6jn" Jul 13 21:34:23.960865 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:34:23.960843 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3124177d_dc55_454c_b9c9_7a2f3557d4f5.slice/crio-6ae3ec5efddef06849688ca3f3208de34ba47453b8887c7f881291a82afcf1a6 WatchSource:0}: Error finding container 6ae3ec5efddef06849688ca3f3208de34ba47453b8887c7f881291a82afcf1a6: Status 404 returned error can't find the container with id 6ae3ec5efddef06849688ca3f3208de34ba47453b8887c7f881291a82afcf1a6 Jul 13 21:34:24.011170 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.011142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:34:24.013684 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.013656 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de0238bd-168e-48aa-9f7c-d8cdea513774-metrics-tls\") pod \"dns-default-4ddkh\" (UID: \"de0238bd-168e-48aa-9f7c-d8cdea513774\") " pod="openshift-dns/dns-default-4ddkh" Jul 13 21:34:24.059020 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.058994 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj"] Jul 13 21:34:24.062561 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:34:24.062529 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae395e51_c106_4727_9615_226822487945.slice/crio-ffbcb3528d45053a03e2e3e92d5d50b0486e8f5201c5c18399ebfe2389adcd30 WatchSource:0}: Error finding container ffbcb3528d45053a03e2e3e92d5d50b0486e8f5201c5c18399ebfe2389adcd30: Status 404 returned error can't find the container with id ffbcb3528d45053a03e2e3e92d5d50b0486e8f5201c5c18399ebfe2389adcd30 Jul 13 21:34:24.208546 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.208513 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l6v89\"" Jul 13 21:34:24.216697 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.216587 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4ddkh" Jul 13 21:34:24.223284 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.223260 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv6jn" event={"ID":"3124177d-dc55-454c-b9c9-7a2f3557d4f5","Type":"ContainerStarted","Data":"6ae3ec5efddef06849688ca3f3208de34ba47453b8887c7f881291a82afcf1a6"} Jul 13 21:34:24.225473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.225348 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" event={"ID":"ae395e51-c106-4727-9615-226822487945","Type":"ContainerStarted","Data":"37a0b97fa11e95eaf5b62afe5a532705865f29efd8f356ad3538bb9362af91ec"} Jul 13 21:34:24.225473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.225402 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" event={"ID":"ae395e51-c106-4727-9615-226822487945","Type":"ContainerStarted","Data":"e92ac936f35f2e0f7725268f274aa7193b6b973bb077b36ede9aaae03cfb99c4"} Jul 13 21:34:24.225473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.225431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" event={"ID":"ae395e51-c106-4727-9615-226822487945","Type":"ContainerStarted","Data":"ffbcb3528d45053a03e2e3e92d5d50b0486e8f5201c5c18399ebfe2389adcd30"} Jul 13 21:34:24.350358 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:24.350333 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4ddkh"] Jul 13 21:34:24.351321 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:34:24.351299 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0238bd_168e_48aa_9f7c_d8cdea513774.slice/crio-3071ffe6097c70ad2333a71cc39d05571f5b0e216fdb7f2709e6fbd4d32db2f1 WatchSource:0}: Error finding container 3071ffe6097c70ad2333a71cc39d05571f5b0e216fdb7f2709e6fbd4d32db2f1: Status 404 returned error can't find the container with id 3071ffe6097c70ad2333a71cc39d05571f5b0e216fdb7f2709e6fbd4d32db2f1 Jul 13 21:34:25.231334 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:25.231294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4ddkh" event={"ID":"de0238bd-168e-48aa-9f7c-d8cdea513774","Type":"ContainerStarted","Data":"3071ffe6097c70ad2333a71cc39d05571f5b0e216fdb7f2709e6fbd4d32db2f1"} Jul 13 21:34:26.237705 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:26.237667 2583 generic.go:356] "Generic (PLEG): container finished" podID="3124177d-dc55-454c-b9c9-7a2f3557d4f5" containerID="f01c0d2760457c665c1c571aed94dda1cb74444f4e2f3abaa2d99ac419ed8c41" exitCode=0 Jul 13 21:34:26.238122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:26.237804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv6jn" event={"ID":"3124177d-dc55-454c-b9c9-7a2f3557d4f5","Type":"ContainerDied","Data":"f01c0d2760457c665c1c571aed94dda1cb74444f4e2f3abaa2d99ac419ed8c41"} Jul 13 21:34:26.241128 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:26.241094 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" event={"ID":"ae395e51-c106-4727-9615-226822487945","Type":"ContainerStarted","Data":"907108bbf734190ee8abdd8ee3cf7fa1a96b4f48cae17e990b2dbbb1343a76e3"} Jul 13 21:34:26.271566 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:26.271495 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-fwhsj" podStartSLOduration=2.065473212 podStartE2EDuration="3.271473074s" totalImagesPullingTime="1.205999862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:26.270517608 +0000 UTC m=+287.090027450" watchObservedRunningTime="2026-07-13 21:34:26.271473074 +0000 UTC m=+287.090982863" Jul 13 21:34:27.247935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.247898 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv6jn" event={"ID":"3124177d-dc55-454c-b9c9-7a2f3557d4f5","Type":"ContainerStarted","Data":"b0ffb816b44c504085bb6fdec2bd0d4d4c47e3c0ceff04ff6e121ef428d78409"} Jul 13 21:34:27.248426 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.247980 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv6jn" event={"ID":"3124177d-dc55-454c-b9c9-7a2f3557d4f5","Type":"ContainerStarted","Data":"30da128a3e35882892b4a1b09a75745ef195e3bce0e2d7d5ecc4dbac1f751812"} Jul 13 21:34:27.250078 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.250047 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4ddkh" event={"ID":"de0238bd-168e-48aa-9f7c-d8cdea513774","Type":"ContainerStarted","Data":"d31bad8a0314fdaec257b6e6badaa9b1c701c76198c5f3da318757dfb1616306"} Jul 13 21:34:27.250222 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.250111 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4ddkh" event={"ID":"de0238bd-168e-48aa-9f7c-d8cdea513774","Type":"ContainerStarted","Data":"04d6de386afc40792a21fce2ffefb38e577611102ba411d21bb44cbaeec22cd9"} Jul 13 21:34:27.250322 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.250298 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4ddkh" Jul 13 21:34:27.267593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.267553 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rv6jn" podStartSLOduration=2.761290201 podStartE2EDuration="4.267534074s" totalImagesPullingTime="1.430356451s" totalInitContainerRuntime="75.887422ms" isStatefulPod=true podCreationTimestamp="2026-07-13 21:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:27.266078355 +0000 UTC m=+288.085588185" watchObservedRunningTime="2026-07-13 21:34:27.267534074 +0000 UTC m=+288.087043884" Jul 13 21:34:27.282729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:27.282693 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4ddkh" podStartSLOduration=251.967687643 podStartE2EDuration="4m14.282680094s" totalImagesPullingTime="2.314992451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:27.281102591 +0000 UTC m=+288.100612400" watchObservedRunningTime="2026-07-13 21:34:27.282680094 +0000 UTC m=+288.102189893" Jul 13 21:34:28.390300 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.390268 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq"] Jul 13 21:34:28.478525 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.478500 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq"] Jul 13 21:34:28.478673 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.478584 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.481247 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.481217 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 21:34:28.481247 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.481217 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jvh85\"" Jul 13 21:34:28.549299 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.549269 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11d8cb6a-c43e-4c00-a04e-15eb927c8b0f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-q4svq\" (UID: \"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.549431 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.549317 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4dn\" (UniqueName: \"kubernetes.io/projected/11d8cb6a-c43e-4c00-a04e-15eb927c8b0f-kube-api-access-bj4dn\") pod \"monitoring-plugin-5f68df7f58-q4svq\" (UID: \"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.649919 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.649864 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11d8cb6a-c43e-4c00-a04e-15eb927c8b0f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-q4svq\" (UID: \"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.649919 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.649905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4dn\" (UniqueName: \"kubernetes.io/projected/11d8cb6a-c43e-4c00-a04e-15eb927c8b0f-kube-api-access-bj4dn\") pod \"monitoring-plugin-5f68df7f58-q4svq\" (UID: \"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.652223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.652193 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11d8cb6a-c43e-4c00-a04e-15eb927c8b0f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-q4svq\" (UID: \"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.657896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.657870 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4dn\" (UniqueName: \"kubernetes.io/projected/11d8cb6a-c43e-4c00-a04e-15eb927c8b0f-kube-api-access-bj4dn\") pod \"monitoring-plugin-5f68df7f58-q4svq\" (UID: \"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.788866 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.788839 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:28.814123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.814098 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-dd44455bd-45775"] Jul 13 21:34:28.816917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.816900 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.824606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.824584 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 21:34:28.824857 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.824840 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 21:34:28.824955 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.824916 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 21:34:28.826056 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.826036 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jdvnf\"" Jul 13 21:34:28.826148 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.826041 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 21:34:28.826148 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.826044 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 21:34:28.831535 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.831508 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 21:34:28.832558 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.832536 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dd44455bd-45775"] Jul 13 21:34:28.919316 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.919242 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq"] Jul 13 21:34:28.923868 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:34:28.923843 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d8cb6a_c43e_4c00_a04e_15eb927c8b0f.slice/crio-189f3417e827ac6d827aead0b00585e8f6f3773f5cd51af06966bb50b00e97ea WatchSource:0}: Error finding container 189f3417e827ac6d827aead0b00585e8f6f3773f5cd51af06966bb50b00e97ea: Status 404 returned error can't find the container with id 189f3417e827ac6d827aead0b00585e8f6f3773f5cd51af06966bb50b00e97ea Jul 13 21:34:28.953345 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953462 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-federate-client-tls\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953462 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953411 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-telemeter-client-tls\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953576 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953477 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c46k\" (UniqueName: \"kubernetes.io/projected/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-kube-api-access-9c46k\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953576 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-secret-telemeter-client\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953580 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-serving-certs-ca-bundle\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953620 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:28.953682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:28.953670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-metrics-client-ca\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.054632 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054604 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-federate-client-tls\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.054753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054649 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-telemeter-client-tls\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.054753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c46k\" (UniqueName: \"kubernetes.io/projected/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-kube-api-access-9c46k\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.054753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054692 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-secret-telemeter-client\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.054753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054718 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-serving-certs-ca-bundle\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.054753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054743 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.055030 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054806 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-metrics-client-ca\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.055030 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.054857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.055674 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.055648 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-metrics-client-ca\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.055799 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.055648 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-serving-certs-ca-bundle\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.055799 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.055729 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.057108 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.057070 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-federate-client-tls\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.057205 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.057157 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-secret-telemeter-client\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.057385 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.057371 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-telemeter-client-tls\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.057422 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.057395 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.062558 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.062539 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c46k\" (UniqueName: \"kubernetes.io/projected/2aa3e973-b1c9-48cb-8f14-7b879f22ec4c-kube-api-access-9c46k\") pod \"telemeter-client-dd44455bd-45775\" (UID: \"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c\") " pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.126531 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.126504 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" Jul 13 21:34:29.261844 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.261799 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" event={"ID":"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f","Type":"ContainerStarted","Data":"189f3417e827ac6d827aead0b00585e8f6f3773f5cd51af06966bb50b00e97ea"} Jul 13 21:34:29.301507 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:29.301482 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dd44455bd-45775"] Jul 13 21:34:29.303792 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:34:29.303753 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aa3e973_b1c9_48cb_8f14_7b879f22ec4c.slice/crio-3389d4b5604215dc7d802afcd26b181f705478be285149b7d5abe54236ab65d3 WatchSource:0}: Error finding container 3389d4b5604215dc7d802afcd26b181f705478be285149b7d5abe54236ab65d3: Status 404 returned error can't find the container with id 3389d4b5604215dc7d802afcd26b181f705478be285149b7d5abe54236ab65d3 Jul 13 21:34:30.267266 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:30.267225 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" event={"ID":"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c","Type":"ContainerStarted","Data":"3389d4b5604215dc7d802afcd26b181f705478be285149b7d5abe54236ab65d3"} Jul 13 21:34:31.277553 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:31.277510 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" event={"ID":"11d8cb6a-c43e-4c00-a04e-15eb927c8b0f","Type":"ContainerStarted","Data":"32c17cad17beca96f9f8bd946e86ac4ebd1b3494354d266f4fd0cca15833c902"} Jul 13 21:34:31.278118 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:31.277703 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:31.282755 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:31.282734 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" Jul 13 21:34:31.292350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:31.292301 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-q4svq" podStartSLOduration=1.7694954219999999 podStartE2EDuration="3.292286163s" totalImagesPullingTime="1.522790741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:31.291529942 +0000 UTC m=+292.111039753" watchObservedRunningTime="2026-07-13 21:34:31.292286163 +0000 UTC m=+292.111795971" Jul 13 21:34:32.283349 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:32.283267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" event={"ID":"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c","Type":"ContainerStarted","Data":"c6a45af64ee3b067e36ba1d5202c6d478a0e2789bce000bdbe9390f4c1c42e67"} Jul 13 21:34:33.289267 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:33.289226 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" event={"ID":"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c","Type":"ContainerStarted","Data":"3fecc1a6f6b0f6e6549c44229ad53caaa93d5fcc81056ca4db65a684242024e3"} Jul 13 21:34:33.289720 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:33.289282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" event={"ID":"2aa3e973-b1c9-48cb-8f14-7b879f22ec4c","Type":"ContainerStarted","Data":"d1c9000a358c817ee01fc4cd4bc3b6edc0341fccab4dbb12f03639c7da9e9fed"} Jul 13 21:34:33.311272 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:33.311222 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-dd44455bd-45775" podStartSLOduration=1.96256611 podStartE2EDuration="5.311203293s" totalImagesPullingTime="3.348637183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:33.309892505 +0000 UTC m=+294.129402336" watchObservedRunningTime="2026-07-13 21:34:33.311203293 +0000 UTC m=+294.130713101" Jul 13 21:34:35.346530 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:34:35.346482 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-49ngc" podUID="5afe6508-8b7b-4083-ac75-e95e48666e82" Jul 13 21:34:37.257137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:37.257104 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4ddkh" Jul 13 21:34:40.384794 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:40.384658 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:34:40.409883 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:40.384875 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:34:40.409883 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:40.390578 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 21:34:41.321331 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:41.321290 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-7l94x" event={"ID":"57d45916-3720-410a-a967-c448673c7233","Type":"ContainerStarted","Data":"11349370fe014595d6a2bc2c054ca553c0c126396db6235b3e5b1704cd63aabf"} Jul 13 21:34:41.321512 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:41.321468 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:41.335642 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:41.335614 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-7l94x" Jul 13 21:34:41.339133 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:41.339075 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-7l94x" podStartSLOduration=2.272049759 podStartE2EDuration="20.339057283s" totalImagesPullingTime="18.067007524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:41.337399901 +0000 UTC m=+302.156909723" watchObservedRunningTime="2026-07-13 21:34:41.339057283 +0000 UTC m=+302.158567093" Jul 13 21:34:46.346397 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:46.346368 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:34:50.811103 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:34:50.811058 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" podUID="325e76f5-23f6-4be3-980a-c407486f81f8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 13 21:35:00.811340 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:00.811303 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" podUID="325e76f5-23f6-4be3-980a-c407486f81f8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 13 21:35:10.810454 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:10.810416 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" podUID="325e76f5-23f6-4be3-980a-c407486f81f8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 13 21:35:10.810813 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:10.810476 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" Jul 13 21:35:10.810903 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:10.810875 2583 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c09f7facf77c8c7b4005c4e723c251a1632999cf4dbf4f3800d3cc5d185099ee"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 13 21:35:10.810942 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:10.810922 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" podUID="325e76f5-23f6-4be3-980a-c407486f81f8" containerName="service-proxy" containerID="cri-o://c09f7facf77c8c7b4005c4e723c251a1632999cf4dbf4f3800d3cc5d185099ee" gracePeriod=30 Jul 13 21:35:11.419157 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:11.419126 2583 generic.go:356] "Generic (PLEG): container finished" podID="325e76f5-23f6-4be3-980a-c407486f81f8" containerID="c09f7facf77c8c7b4005c4e723c251a1632999cf4dbf4f3800d3cc5d185099ee" exitCode=2 Jul 13 21:35:11.419324 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:11.419204 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" event={"ID":"325e76f5-23f6-4be3-980a-c407486f81f8","Type":"ContainerDied","Data":"c09f7facf77c8c7b4005c4e723c251a1632999cf4dbf4f3800d3cc5d185099ee"} Jul 13 21:35:11.419324 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:11.419258 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-646668dfff-2hvlw" event={"ID":"325e76f5-23f6-4be3-980a-c407486f81f8","Type":"ContainerStarted","Data":"79a32271892b5cb80a83184a822c912fbe1060a91f9465dce77e195c4cd297d7"} Jul 13 21:35:53.550876 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:53.550846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:35:53.552990 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:53.552973 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5afe6508-8b7b-4083-ac75-e95e48666e82-metrics-certs\") pod \"network-metrics-daemon-49ngc\" (UID: \"5afe6508-8b7b-4083-ac75-e95e48666e82\") " pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:35:53.849835 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:53.849747 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89gzj\"" Jul 13 21:35:53.857795 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:53.857758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49ngc" Jul 13 21:35:53.971877 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:53.971847 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-49ngc"] Jul 13 21:35:53.974929 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:35:53.974898 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5afe6508_8b7b_4083_ac75_e95e48666e82.slice/crio-54e7dcdc500aad9afe67b7cfda434f43c18e69692dd811ba8d4a2550cd6f7109 WatchSource:0}: Error finding container 54e7dcdc500aad9afe67b7cfda434f43c18e69692dd811ba8d4a2550cd6f7109: Status 404 returned error can't find the container with id 54e7dcdc500aad9afe67b7cfda434f43c18e69692dd811ba8d4a2550cd6f7109 Jul 13 21:35:53.976653 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:53.976635 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:35:54.566065 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:54.566027 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-49ngc" event={"ID":"5afe6508-8b7b-4083-ac75-e95e48666e82","Type":"ContainerStarted","Data":"54e7dcdc500aad9afe67b7cfda434f43c18e69692dd811ba8d4a2550cd6f7109"} Jul 13 21:35:55.571738 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:55.571695 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-49ngc" event={"ID":"5afe6508-8b7b-4083-ac75-e95e48666e82","Type":"ContainerStarted","Data":"461ac9217536614e6e63fc3d2c438af7564159890f6093a4786e9dd3d815ad87"} Jul 13 21:35:55.572142 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:55.571758 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-49ngc" event={"ID":"5afe6508-8b7b-4083-ac75-e95e48666e82","Type":"ContainerStarted","Data":"481cd2bf4f7efbb0fd8dfbbd8f97e9a335dcf8d7078b4f4372f70f8c7d609cdf"} Jul 13 21:35:55.587014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:35:55.586964 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-49ngc" podStartSLOduration=374.732813854 podStartE2EDuration="6m15.586946861s" totalImagesPullingTime="854.133007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:35:55.586121373 +0000 UTC m=+376.405631198" watchObservedRunningTime="2026-07-13 21:35:55.586946861 +0000 UTC m=+376.406456669" Jul 13 21:36:50.969053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:50.969022 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p"] Jul 13 21:36:50.970969 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:50.970954 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:50.973258 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:50.973231 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 21:36:50.973371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:50.973337 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tzhx5\"" Jul 13 21:36:50.973371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:50.973355 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 21:36:50.979431 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:50.979412 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p"] Jul 13 21:36:51.008938 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.008919 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.009044 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.008944 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.009044 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.008978 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97pgr\" (UniqueName: \"kubernetes.io/projected/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-kube-api-access-97pgr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.109741 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.109710 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97pgr\" (UniqueName: \"kubernetes.io/projected/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-kube-api-access-97pgr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.109828 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.109782 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.109828 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.109802 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.110122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.110107 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.110161 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.110134 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.117586 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.117565 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97pgr\" (UniqueName: \"kubernetes.io/projected/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-kube-api-access-97pgr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.279699 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.279647 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:36:51.396020 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.395994 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p"] Jul 13 21:36:51.398447 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:36:51.398421 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f154c2_30f6_43aa_ad3c_f489fb867ae9.slice/crio-dfb8dda59b6eb3f2641bbfacb29275aab9348d853f72c6e588c46d002fce6164 WatchSource:0}: Error finding container dfb8dda59b6eb3f2641bbfacb29275aab9348d853f72c6e588c46d002fce6164: Status 404 returned error can't find the container with id dfb8dda59b6eb3f2641bbfacb29275aab9348d853f72c6e588c46d002fce6164 Jul 13 21:36:51.755612 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:51.755583 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" event={"ID":"c0f154c2-30f6-43aa-ad3c-f489fb867ae9","Type":"ContainerStarted","Data":"dfb8dda59b6eb3f2641bbfacb29275aab9348d853f72c6e588c46d002fce6164"} Jul 13 21:36:58.781983 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:58.781953 2583 generic.go:356] "Generic (PLEG): container finished" podID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerID="7e729800a66a4ff704e84edf7712816bada0ff62f3d8bf96ea2f02314d20f13f" exitCode=0 Jul 13 21:36:58.782321 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:36:58.782020 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" event={"ID":"c0f154c2-30f6-43aa-ad3c-f489fb867ae9","Type":"ContainerDied","Data":"7e729800a66a4ff704e84edf7712816bada0ff62f3d8bf96ea2f02314d20f13f"} Jul 13 21:37:00.793466 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:00.793429 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" event={"ID":"c0f154c2-30f6-43aa-ad3c-f489fb867ae9","Type":"ContainerStarted","Data":"cfd888b5d380eb7510063b8bafca9ca34101a6de061ae2306b15048007cdf20a"} Jul 13 21:37:01.798895 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:01.798811 2583 generic.go:356] "Generic (PLEG): container finished" podID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerID="cfd888b5d380eb7510063b8bafca9ca34101a6de061ae2306b15048007cdf20a" exitCode=0 Jul 13 21:37:01.799328 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:01.798903 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" event={"ID":"c0f154c2-30f6-43aa-ad3c-f489fb867ae9","Type":"ContainerDied","Data":"cfd888b5d380eb7510063b8bafca9ca34101a6de061ae2306b15048007cdf20a"} Jul 13 21:37:09.834643 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:09.834602 2583 generic.go:356] "Generic (PLEG): container finished" podID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerID="a13da9a9410cab126d6b3e62cb1319ae45574dbdcdc207c9aeb830a19be2fc92" exitCode=0 Jul 13 21:37:09.835011 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:09.834665 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" event={"ID":"c0f154c2-30f6-43aa-ad3c-f489fb867ae9","Type":"ContainerDied","Data":"a13da9a9410cab126d6b3e62cb1319ae45574dbdcdc207c9aeb830a19be2fc92"} Jul 13 21:37:10.953938 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:10.953914 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:37:11.059137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.059114 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-bundle\") pod \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " Jul 13 21:37:11.059252 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.059148 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-util\") pod \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " Jul 13 21:37:11.059252 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.059198 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97pgr\" (UniqueName: \"kubernetes.io/projected/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-kube-api-access-97pgr\") pod \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\" (UID: \"c0f154c2-30f6-43aa-ad3c-f489fb867ae9\") " Jul 13 21:37:11.059724 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.059699 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-bundle" (OuterVolumeSpecName: "bundle") pod "c0f154c2-30f6-43aa-ad3c-f489fb867ae9" (UID: "c0f154c2-30f6-43aa-ad3c-f489fb867ae9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:37:11.061299 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.061277 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-kube-api-access-97pgr" (OuterVolumeSpecName: "kube-api-access-97pgr") pod "c0f154c2-30f6-43aa-ad3c-f489fb867ae9" (UID: "c0f154c2-30f6-43aa-ad3c-f489fb867ae9"). InnerVolumeSpecName "kube-api-access-97pgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:37:11.063708 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.063684 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-util" (OuterVolumeSpecName: "util") pod "c0f154c2-30f6-43aa-ad3c-f489fb867ae9" (UID: "c0f154c2-30f6-43aa-ad3c-f489fb867ae9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:37:11.159762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.159711 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:11.159762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.159733 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97pgr\" (UniqueName: \"kubernetes.io/projected/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-kube-api-access-97pgr\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:11.159762 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.159743 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f154c2-30f6-43aa-ad3c-f489fb867ae9-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:11.844032 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.843992 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" event={"ID":"c0f154c2-30f6-43aa-ad3c-f489fb867ae9","Type":"ContainerDied","Data":"dfb8dda59b6eb3f2641bbfacb29275aab9348d853f72c6e588c46d002fce6164"} Jul 13 21:37:11.844032 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.844031 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n628p" Jul 13 21:37:11.844224 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:11.844054 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfb8dda59b6eb3f2641bbfacb29275aab9348d853f72c6e588c46d002fce6164" Jul 13 21:37:18.983434 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983405 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch"] Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983672 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="util" Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983682 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="util" Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983700 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="extract" Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983706 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="extract" Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983720 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="pull" Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983725 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="pull" Jul 13 21:37:18.983904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.983762 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f154c2-30f6-43aa-ad3c-f489fb867ae9" containerName="extract" Jul 13 21:37:18.988371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.988353 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:18.990904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.990881 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-6c82x\"" Jul 13 21:37:18.990904 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.990890 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 21:37:18.991068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.990961 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:37:18.999482 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:18.999464 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch"] Jul 13 21:37:19.114221 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.114197 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2vkw\" (UniqueName: \"kubernetes.io/projected/3c63e6cc-fd1c-4da1-a70f-59ac09e64422-kube-api-access-r2vkw\") pod \"cert-manager-operator-controller-manager-55c444bf86-mvhch\" (UID: \"3c63e6cc-fd1c-4da1-a70f-59ac09e64422\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.114329 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.114243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c63e6cc-fd1c-4da1-a70f-59ac09e64422-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mvhch\" (UID: \"3c63e6cc-fd1c-4da1-a70f-59ac09e64422\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.214879 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.214859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2vkw\" (UniqueName: \"kubernetes.io/projected/3c63e6cc-fd1c-4da1-a70f-59ac09e64422-kube-api-access-r2vkw\") pod \"cert-manager-operator-controller-manager-55c444bf86-mvhch\" (UID: \"3c63e6cc-fd1c-4da1-a70f-59ac09e64422\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.214961 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.214905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c63e6cc-fd1c-4da1-a70f-59ac09e64422-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mvhch\" (UID: \"3c63e6cc-fd1c-4da1-a70f-59ac09e64422\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.215185 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.215171 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c63e6cc-fd1c-4da1-a70f-59ac09e64422-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mvhch\" (UID: \"3c63e6cc-fd1c-4da1-a70f-59ac09e64422\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.222912 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.222889 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2vkw\" (UniqueName: \"kubernetes.io/projected/3c63e6cc-fd1c-4da1-a70f-59ac09e64422-kube-api-access-r2vkw\") pod \"cert-manager-operator-controller-manager-55c444bf86-mvhch\" (UID: \"3c63e6cc-fd1c-4da1-a70f-59ac09e64422\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.296580 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.296530 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" Jul 13 21:37:19.418576 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.418435 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch"] Jul 13 21:37:19.423007 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:37:19.422979 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c63e6cc_fd1c_4da1_a70f_59ac09e64422.slice/crio-89e3f2e867088b6ec93b1d0bf4502fe47229a2792ccaad1395215d7ab2b26f19 WatchSource:0}: Error finding container 89e3f2e867088b6ec93b1d0bf4502fe47229a2792ccaad1395215d7ab2b26f19: Status 404 returned error can't find the container with id 89e3f2e867088b6ec93b1d0bf4502fe47229a2792ccaad1395215d7ab2b26f19 Jul 13 21:37:19.879237 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:19.879202 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" event={"ID":"3c63e6cc-fd1c-4da1-a70f-59ac09e64422","Type":"ContainerStarted","Data":"89e3f2e867088b6ec93b1d0bf4502fe47229a2792ccaad1395215d7ab2b26f19"} Jul 13 21:37:21.888194 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:21.888158 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" event={"ID":"3c63e6cc-fd1c-4da1-a70f-59ac09e64422","Type":"ContainerStarted","Data":"cd2e893f6d305342706f89ae1e32698ddd7e384475e94a63ed03a0c1bfaaac80"} Jul 13 21:37:21.908701 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:21.908656 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mvhch" podStartSLOduration=2.454786899 podStartE2EDuration="3.908643481s" totalImagesPullingTime="1.453856582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:21.90651977 +0000 UTC m=+462.726029578" watchObservedRunningTime="2026-07-13 21:37:21.908643481 +0000 UTC m=+462.728153289" Jul 13 21:37:22.887123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.887089 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh"] Jul 13 21:37:22.890411 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.890390 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:22.893040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.893017 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 21:37:22.893987 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.893970 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tzhx5\"" Jul 13 21:37:22.894049 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.893979 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 21:37:22.901903 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.901885 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh"] Jul 13 21:37:22.940673 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.940649 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d59k6\" (UniqueName: \"kubernetes.io/projected/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-kube-api-access-d59k6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:22.940768 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.940701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:22.940768 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:22.940748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.042089 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.042033 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d59k6\" (UniqueName: \"kubernetes.io/projected/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-kube-api-access-d59k6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.042211 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.042106 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.042274 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.042247 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.042375 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.042360 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.042497 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.042481 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.051264 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.051242 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d59k6\" (UniqueName: \"kubernetes.io/projected/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-kube-api-access-d59k6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.198953 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.198933 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:23.319498 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.319474 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh"] Jul 13 21:37:23.321887 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:37:23.321857 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e4aa6f_f43c_471f_aada_cc4c0bcc05cc.slice/crio-8b0e9433c3623b8a2244d715cb65c6b1d4a2135ea6b6a91cca4681d55b150a7f WatchSource:0}: Error finding container 8b0e9433c3623b8a2244d715cb65c6b1d4a2135ea6b6a91cca4681d55b150a7f: Status 404 returned error can't find the container with id 8b0e9433c3623b8a2244d715cb65c6b1d4a2135ea6b6a91cca4681d55b150a7f Jul 13 21:37:23.897496 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.897463 2583 generic.go:356] "Generic (PLEG): container finished" podID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerID="ea3a0d4f94cb49665dd6e208edeeaf48d2cb2a1336831652008dfc4fd8c9b6a1" exitCode=0 Jul 13 21:37:23.897899 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.897549 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" event={"ID":"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc","Type":"ContainerDied","Data":"ea3a0d4f94cb49665dd6e208edeeaf48d2cb2a1336831652008dfc4fd8c9b6a1"} Jul 13 21:37:23.897899 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:23.897605 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" event={"ID":"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc","Type":"ContainerStarted","Data":"8b0e9433c3623b8a2244d715cb65c6b1d4a2135ea6b6a91cca4681d55b150a7f"} Jul 13 21:37:26.911086 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:26.911050 2583 generic.go:356] "Generic (PLEG): container finished" podID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerID="52054eee78a51855512630a4f76310f5b9e5956c1fbb4128ea420eea3b985138" exitCode=0 Jul 13 21:37:26.911563 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:26.911106 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" event={"ID":"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc","Type":"ContainerDied","Data":"52054eee78a51855512630a4f76310f5b9e5956c1fbb4128ea420eea3b985138"} Jul 13 21:37:27.916945 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:27.916907 2583 generic.go:356] "Generic (PLEG): container finished" podID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerID="a36e364a854235dd0a97934e537621fff8f2bc7f104a5121e5c1af4e5a125f1f" exitCode=0 Jul 13 21:37:27.917305 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:27.916956 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" event={"ID":"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc","Type":"ContainerDied","Data":"a36e364a854235dd0a97934e537621fff8f2bc7f104a5121e5c1af4e5a125f1f"} Jul 13 21:37:29.043103 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.043082 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:29.187311 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.187286 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-bundle\") pod \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " Jul 13 21:37:29.187466 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.187324 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-util\") pod \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " Jul 13 21:37:29.187466 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.187346 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d59k6\" (UniqueName: \"kubernetes.io/projected/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-kube-api-access-d59k6\") pod \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\" (UID: \"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc\") " Jul 13 21:37:29.187702 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.187674 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-bundle" (OuterVolumeSpecName: "bundle") pod "76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" (UID: "76e4aa6f-f43c-471f-aada-cc4c0bcc05cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:37:29.189344 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.189316 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-kube-api-access-d59k6" (OuterVolumeSpecName: "kube-api-access-d59k6") pod "76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" (UID: "76e4aa6f-f43c-471f-aada-cc4c0bcc05cc"). InnerVolumeSpecName "kube-api-access-d59k6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:37:29.191917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.191884 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-util" (OuterVolumeSpecName: "util") pod "76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" (UID: "76e4aa6f-f43c-471f-aada-cc4c0bcc05cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:37:29.288368 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.288339 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:29.288368 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.288362 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:29.288368 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.288373 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d59k6\" (UniqueName: \"kubernetes.io/projected/76e4aa6f-f43c-471f-aada-cc4c0bcc05cc-kube-api-access-d59k6\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:29.926438 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.926413 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" Jul 13 21:37:29.926580 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.926412 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcmxvh" event={"ID":"76e4aa6f-f43c-471f-aada-cc4c0bcc05cc","Type":"ContainerDied","Data":"8b0e9433c3623b8a2244d715cb65c6b1d4a2135ea6b6a91cca4681d55b150a7f"} Jul 13 21:37:29.926580 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:29.926527 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b0e9433c3623b8a2244d715cb65c6b1d4a2135ea6b6a91cca4681d55b150a7f" Jul 13 21:37:35.046714 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.046678 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2"] Jul 13 21:37:35.047164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047050 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="util" Jul 13 21:37:35.047164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047065 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="util" Jul 13 21:37:35.047164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047078 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="pull" Jul 13 21:37:35.047164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047086 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="pull" Jul 13 21:37:35.047164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047096 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="extract" Jul 13 21:37:35.047164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047104 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="extract" Jul 13 21:37:35.047459 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.047181 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e4aa6f-f43c-471f-aada-cc4c0bcc05cc" containerName="extract" Jul 13 21:37:35.049938 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.049919 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.052163 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.052143 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:37:35.052991 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.052976 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 21:37:35.052991 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.052981 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ftdwg\"" Jul 13 21:37:35.059641 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.059620 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2"] Jul 13 21:37:35.133002 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.132977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szbn9\" (UniqueName: \"kubernetes.io/projected/66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6-kube-api-access-szbn9\") pod \"openshift-lws-operator-fd8ccff4c-h9qz2\" (UID: \"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.133119 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.133006 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h9qz2\" (UID: \"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.233719 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.233690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szbn9\" (UniqueName: \"kubernetes.io/projected/66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6-kube-api-access-szbn9\") pod \"openshift-lws-operator-fd8ccff4c-h9qz2\" (UID: \"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.233871 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.233723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h9qz2\" (UID: \"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.234086 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.234067 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h9qz2\" (UID: \"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.241975 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.241954 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szbn9\" (UniqueName: \"kubernetes.io/projected/66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6-kube-api-access-szbn9\") pod \"openshift-lws-operator-fd8ccff4c-h9qz2\" (UID: \"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.359180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.359119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" Jul 13 21:37:35.479059 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.479020 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2"] Jul 13 21:37:35.482094 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:37:35.482069 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66dc8bfa_4f8b_4a83_ba22_7fdf8aa5b8f6.slice/crio-44f98ca23472fa73c5ccecefdb520ae945bc1b53f79c37c4c135d6b41eb84d56 WatchSource:0}: Error finding container 44f98ca23472fa73c5ccecefdb520ae945bc1b53f79c37c4c135d6b41eb84d56: Status 404 returned error can't find the container with id 44f98ca23472fa73c5ccecefdb520ae945bc1b53f79c37c4c135d6b41eb84d56 Jul 13 21:37:35.949549 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:35.949518 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" event={"ID":"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6","Type":"ContainerStarted","Data":"44f98ca23472fa73c5ccecefdb520ae945bc1b53f79c37c4c135d6b41eb84d56"} Jul 13 21:37:37.959698 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:37.959661 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" event={"ID":"66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6","Type":"ContainerStarted","Data":"a4ffa7ee49ef6270f98de5e0c6aa03e4633334bb63caabb774893e438c5843a1"} Jul 13 21:37:37.976633 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:37.976580 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h9qz2" podStartSLOduration=1.296169673 podStartE2EDuration="2.976561718s" totalImagesPullingTime="1.680392045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:37.974818036 +0000 UTC m=+478.794327846" watchObservedRunningTime="2026-07-13 21:37:37.976561718 +0000 UTC m=+478.796071525" Jul 13 21:37:40.921401 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:40.921361 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp"] Jul 13 21:37:40.924917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:40.924894 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:40.927267 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:40.927249 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 21:37:40.927420 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:40.927403 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 21:37:40.927493 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:40.927479 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tzhx5\"" Jul 13 21:37:40.933636 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:40.933616 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp"] Jul 13 21:37:41.075706 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.075678 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.075869 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.075757 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pv4k\" (UniqueName: \"kubernetes.io/projected/de1d6d41-545d-4740-8b90-230a944c6519-kube-api-access-5pv4k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.075869 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.075809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.177020 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.176994 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.177152 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.177054 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pv4k\" (UniqueName: \"kubernetes.io/projected/de1d6d41-545d-4740-8b90-230a944c6519-kube-api-access-5pv4k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.177152 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.177075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.177433 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.177418 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.177473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.177418 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.185872 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.185841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pv4k\" (UniqueName: \"kubernetes.io/projected/de1d6d41-545d-4740-8b90-230a944c6519-kube-api-access-5pv4k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.233908 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.233867 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:41.363479 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.363447 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp"] Jul 13 21:37:41.366312 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:37:41.366287 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1d6d41_545d_4740_8b90_230a944c6519.slice/crio-885b48d5ab22070799f0ebd5eb0837ff7181516986c96c6afe506e839e0042c7 WatchSource:0}: Error finding container 885b48d5ab22070799f0ebd5eb0837ff7181516986c96c6afe506e839e0042c7: Status 404 returned error can't find the container with id 885b48d5ab22070799f0ebd5eb0837ff7181516986c96c6afe506e839e0042c7 Jul 13 21:37:41.977680 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.977647 2583 generic.go:356] "Generic (PLEG): container finished" podID="de1d6d41-545d-4740-8b90-230a944c6519" containerID="fe5247ea1afba871b8761445fe941b408178792a7ceaeeed3a72ff66e2e07122" exitCode=0 Jul 13 21:37:41.978066 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.977689 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" event={"ID":"de1d6d41-545d-4740-8b90-230a944c6519","Type":"ContainerDied","Data":"fe5247ea1afba871b8761445fe941b408178792a7ceaeeed3a72ff66e2e07122"} Jul 13 21:37:41.978066 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:41.977723 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" event={"ID":"de1d6d41-545d-4740-8b90-230a944c6519","Type":"ContainerStarted","Data":"885b48d5ab22070799f0ebd5eb0837ff7181516986c96c6afe506e839e0042c7"} Jul 13 21:37:43.987169 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:43.987132 2583 generic.go:356] "Generic (PLEG): container finished" podID="de1d6d41-545d-4740-8b90-230a944c6519" containerID="3792c825507110c10c5020ddc95f376fb1362fcc3d412cb81497c2510497d074" exitCode=0 Jul 13 21:37:43.987570 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:43.987203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" event={"ID":"de1d6d41-545d-4740-8b90-230a944c6519","Type":"ContainerDied","Data":"3792c825507110c10c5020ddc95f376fb1362fcc3d412cb81497c2510497d074"} Jul 13 21:37:44.993387 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:44.993353 2583 generic.go:356] "Generic (PLEG): container finished" podID="de1d6d41-545d-4740-8b90-230a944c6519" containerID="f64b8319166142a2c3b153c134c0e3f4906dd18189a1a6b14bd3d75d331f0dab" exitCode=0 Jul 13 21:37:44.993850 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:44.993413 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" event={"ID":"de1d6d41-545d-4740-8b90-230a944c6519","Type":"ContainerDied","Data":"f64b8319166142a2c3b153c134c0e3f4906dd18189a1a6b14bd3d75d331f0dab"} Jul 13 21:37:46.118229 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.118205 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:46.219056 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.219021 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-util\") pod \"de1d6d41-545d-4740-8b90-230a944c6519\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " Jul 13 21:37:46.219264 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.219093 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-bundle\") pod \"de1d6d41-545d-4740-8b90-230a944c6519\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " Jul 13 21:37:46.219264 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.219135 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pv4k\" (UniqueName: \"kubernetes.io/projected/de1d6d41-545d-4740-8b90-230a944c6519-kube-api-access-5pv4k\") pod \"de1d6d41-545d-4740-8b90-230a944c6519\" (UID: \"de1d6d41-545d-4740-8b90-230a944c6519\") " Jul 13 21:37:46.219857 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.219832 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-bundle" (OuterVolumeSpecName: "bundle") pod "de1d6d41-545d-4740-8b90-230a944c6519" (UID: "de1d6d41-545d-4740-8b90-230a944c6519"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:37:46.221211 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.221184 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de1d6d41-545d-4740-8b90-230a944c6519-kube-api-access-5pv4k" (OuterVolumeSpecName: "kube-api-access-5pv4k") pod "de1d6d41-545d-4740-8b90-230a944c6519" (UID: "de1d6d41-545d-4740-8b90-230a944c6519"). InnerVolumeSpecName "kube-api-access-5pv4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:37:46.224619 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.224596 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-util" (OuterVolumeSpecName: "util") pod "de1d6d41-545d-4740-8b90-230a944c6519" (UID: "de1d6d41-545d-4740-8b90-230a944c6519"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:37:46.320100 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.320013 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:46.320100 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.320043 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de1d6d41-545d-4740-8b90-230a944c6519-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:46.320100 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:46.320052 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pv4k\" (UniqueName: \"kubernetes.io/projected/de1d6d41-545d-4740-8b90-230a944c6519-kube-api-access-5pv4k\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:37:47.003216 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:47.003186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" event={"ID":"de1d6d41-545d-4740-8b90-230a944c6519","Type":"ContainerDied","Data":"885b48d5ab22070799f0ebd5eb0837ff7181516986c96c6afe506e839e0042c7"} Jul 13 21:37:47.003216 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:47.003218 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvntqp" Jul 13 21:37:47.003408 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:47.003232 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="885b48d5ab22070799f0ebd5eb0837ff7181516986c96c6afe506e839e0042c7" Jul 13 21:37:58.867037 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867003 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc"] Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867265 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="util" Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867279 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="util" Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867292 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="pull" Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867298 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="pull" Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867309 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="extract" Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867314 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="extract" Jul 13 21:37:58.867425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.867365 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="de1d6d41-545d-4740-8b90-230a944c6519" containerName="extract" Jul 13 21:37:58.873368 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.873351 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:58.876577 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.876554 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 21:37:58.876683 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.876662 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6rfkx\"" Jul 13 21:37:58.876821 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.876803 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 21:37:58.877304 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.877289 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 21:37:58.886210 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.886192 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 21:37:58.893875 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:58.893858 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc"] Jul 13 21:37:59.004955 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.004927 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-tmp\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.005088 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.004968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-manifests\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.005088 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.004991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.005189 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.005117 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjm6h\" (UniqueName: \"kubernetes.io/projected/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-kube-api-access-xjm6h\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.005189 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.005164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.106424 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.106401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-tmp\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.106515 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.106430 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-manifests\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.106515 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.106446 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.106515 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.106496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjm6h\" (UniqueName: \"kubernetes.io/projected/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-kube-api-access-xjm6h\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.106679 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.106534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.106909 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.106887 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-manifests\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.108524 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.108506 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-tmp\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.108871 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.108851 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.108917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.108883 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.115977 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.115956 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjm6h\" (UniqueName: \"kubernetes.io/projected/06e0de3a-b207-4d77-b51c-2e05e01b3b5c-kube-api-access-xjm6h\") pod \"opendatahub-operator-controller-manager-5db8865497-728pc\" (UID: \"06e0de3a-b207-4d77-b51c-2e05e01b3b5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.183251 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.183229 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:37:59.333028 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:37:59.333006 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc"] Jul 13 21:37:59.335659 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:37:59.335620 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e0de3a_b207_4d77_b51c_2e05e01b3b5c.slice/crio-1b4b108ca8908743201ee68ac858011436ce0b861ec209c25b336903274ccf40 WatchSource:0}: Error finding container 1b4b108ca8908743201ee68ac858011436ce0b861ec209c25b336903274ccf40: Status 404 returned error can't find the container with id 1b4b108ca8908743201ee68ac858011436ce0b861ec209c25b336903274ccf40 Jul 13 21:38:00.058590 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:00.058531 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" event={"ID":"06e0de3a-b207-4d77-b51c-2e05e01b3b5c","Type":"ContainerStarted","Data":"1b4b108ca8908743201ee68ac858011436ce0b861ec209c25b336903274ccf40"} Jul 13 21:38:03.077220 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:03.077189 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" event={"ID":"06e0de3a-b207-4d77-b51c-2e05e01b3b5c","Type":"ContainerStarted","Data":"2e6b23b3793e66920c2fd8d34642ee022028ed0d9613f96137c76facbd36af05"} Jul 13 21:38:04.082927 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:04.082893 2583 generic.go:356] "Generic (PLEG): container finished" podID="06e0de3a-b207-4d77-b51c-2e05e01b3b5c" containerID="2e6b23b3793e66920c2fd8d34642ee022028ed0d9613f96137c76facbd36af05" exitCode=0 Jul 13 21:38:04.083279 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:04.082956 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" event={"ID":"06e0de3a-b207-4d77-b51c-2e05e01b3b5c","Type":"ContainerDied","Data":"2e6b23b3793e66920c2fd8d34642ee022028ed0d9613f96137c76facbd36af05"} Jul 13 21:38:05.089703 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:05.089666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" event={"ID":"06e0de3a-b207-4d77-b51c-2e05e01b3b5c","Type":"ContainerStarted","Data":"9032ea2f6b06adbc4044bb04e51bc8483075e56c7c2bff2df187c0b97159ae81"} Jul 13 21:38:05.090087 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:05.089768 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:38:05.110928 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:05.110853 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" podStartSLOduration=2.60049312 podStartE2EDuration="7.110834082s" totalImagesPullingTime="3.442489003s" totalInitContainerRuntime="1.067851959s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:05.108998009 +0000 UTC m=+505.928507819" watchObservedRunningTime="2026-07-13 21:38:05.110834082 +0000 UTC m=+505.930343892" Jul 13 21:38:16.096617 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:16.096587 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-728pc" Jul 13 21:38:18.858210 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.858172 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bff766844-kh9fs"] Jul 13 21:38:18.866024 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.865996 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:18.869209 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.869186 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 21:38:18.870156 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.870139 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 21:38:18.870156 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.870145 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 21:38:18.870272 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.870179 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tk8zb\"" Jul 13 21:38:18.870751 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.870737 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 21:38:18.878900 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.878880 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bff766844-kh9fs"] Jul 13 21:38:18.951094 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.951062 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d654a0c-921f-475d-b656-2795bd12952a-tmp\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:18.951223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.951111 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d654a0c-921f-475d-b656-2795bd12952a-tls-certs\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:18.951223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:18.951134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29c9h\" (UniqueName: \"kubernetes.io/projected/3d654a0c-921f-475d-b656-2795bd12952a-kube-api-access-29c9h\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.051751 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.051721 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d654a0c-921f-475d-b656-2795bd12952a-tmp\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.051912 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.051768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d654a0c-921f-475d-b656-2795bd12952a-tls-certs\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.051912 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.051822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29c9h\" (UniqueName: \"kubernetes.io/projected/3d654a0c-921f-475d-b656-2795bd12952a-kube-api-access-29c9h\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.053947 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.053915 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d654a0c-921f-475d-b656-2795bd12952a-tmp\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.054204 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.054183 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d654a0c-921f-475d-b656-2795bd12952a-tls-certs\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.070102 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.070082 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29c9h\" (UniqueName: \"kubernetes.io/projected/3d654a0c-921f-475d-b656-2795bd12952a-kube-api-access-29c9h\") pod \"kube-auth-proxy-6bff766844-kh9fs\" (UID: \"3d654a0c-921f-475d-b656-2795bd12952a\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.175094 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.175066 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" Jul 13 21:38:19.181242 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.181215 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-57hdq"] Jul 13 21:38:19.185803 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.185767 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.189330 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.189060 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 13 21:38:19.189330 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.189109 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5lf2g\"" Jul 13 21:38:19.217234 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.217206 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-57hdq"] Jul 13 21:38:19.253249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.253221 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d3a8120-670f-41a4-bfcb-1cce715a150b-cert\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.253389 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.253267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpgrl\" (UniqueName: \"kubernetes.io/projected/7d3a8120-670f-41a4-bfcb-1cce715a150b-kube-api-access-wpgrl\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.324911 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.324885 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bff766844-kh9fs"] Jul 13 21:38:19.328253 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:19.328215 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d654a0c_921f_475d_b656_2795bd12952a.slice/crio-02059965ae88dcb64a81f0a98fc660b406d163099af7b8bc5f5993cf16be0f9c WatchSource:0}: Error finding container 02059965ae88dcb64a81f0a98fc660b406d163099af7b8bc5f5993cf16be0f9c: Status 404 returned error can't find the container with id 02059965ae88dcb64a81f0a98fc660b406d163099af7b8bc5f5993cf16be0f9c Jul 13 21:38:19.354445 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.354420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpgrl\" (UniqueName: \"kubernetes.io/projected/7d3a8120-670f-41a4-bfcb-1cce715a150b-kube-api-access-wpgrl\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.354557 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.354532 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d3a8120-670f-41a4-bfcb-1cce715a150b-cert\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.354664 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:19.354621 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 13 21:38:19.354724 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:19.354674 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d3a8120-670f-41a4-bfcb-1cce715a150b-cert podName:7d3a8120-670f-41a4-bfcb-1cce715a150b nodeName:}" failed. No retries permitted until 2026-07-13 21:38:19.854658564 +0000 UTC m=+520.674168354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d3a8120-670f-41a4-bfcb-1cce715a150b-cert") pod "odh-model-controller-7998c67dc4-57hdq" (UID: "7d3a8120-670f-41a4-bfcb-1cce715a150b") : secret "odh-model-controller-webhook-cert" not found Jul 13 21:38:19.369164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.369138 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpgrl\" (UniqueName: \"kubernetes.io/projected/7d3a8120-670f-41a4-bfcb-1cce715a150b-kube-api-access-wpgrl\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.858792 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.858748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d3a8120-670f-41a4-bfcb-1cce715a150b-cert\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:19.861040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:19.861023 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d3a8120-670f-41a4-bfcb-1cce715a150b-cert\") pod \"odh-model-controller-7998c67dc4-57hdq\" (UID: \"7d3a8120-670f-41a4-bfcb-1cce715a150b\") " pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:20.108613 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.108569 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:20.159303 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.159271 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" event={"ID":"3d654a0c-921f-475d-b656-2795bd12952a","Type":"ContainerStarted","Data":"02059965ae88dcb64a81f0a98fc660b406d163099af7b8bc5f5993cf16be0f9c"} Jul 13 21:38:20.268270 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.268238 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-57hdq"] Jul 13 21:38:20.271702 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.271677 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9swdn"] Jul 13 21:38:20.273107 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:20.273079 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d3a8120_670f_41a4_bfcb_1cce715a150b.slice/crio-5a06cd8654e15095bb4c77b043174a3c473554fd121b3ce1efe182266d0dd560 WatchSource:0}: Error finding container 5a06cd8654e15095bb4c77b043174a3c473554fd121b3ce1efe182266d0dd560: Status 404 returned error can't find the container with id 5a06cd8654e15095bb4c77b043174a3c473554fd121b3ce1efe182266d0dd560 Jul 13 21:38:20.276497 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.276355 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.278698 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.278677 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 21:38:20.287404 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.287380 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9swdn"] Jul 13 21:38:20.363054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.362973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-tls-certs\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.363054 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.363008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhl7q\" (UniqueName: \"kubernetes.io/projected/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-kube-api-access-lhl7q\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.399745 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.399692 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs"] Jul 13 21:38:20.403449 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.403425 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.405746 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.405725 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 21:38:20.405896 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.405756 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 21:38:20.406211 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.406193 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 21:38:20.406350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.406240 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j9p4l\"" Jul 13 21:38:20.406491 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.406444 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 21:38:20.414477 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.414457 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs"] Jul 13 21:38:20.463990 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.463960 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.463996 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91425bbc-3037-4fb6-97e3-6bb657d0db32-local-certs\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464059 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-token\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464116 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-tls-certs\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.464350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhl7q\" (UniqueName: \"kubernetes.io/projected/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-kube-api-access-lhl7q\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.464350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-cacerts\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464350 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:20.464233 2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 13 21:38:20.464350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464252 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kpgm\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-kube-api-access-8kpgm\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.464290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.464350 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:20.464307 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-tls-certs podName:cd2ea4b2-0027-4cd7-8e29-e36db1367a36 nodeName:}" failed. No retries permitted until 2026-07-13 21:38:20.964285146 +0000 UTC m=+521.783794946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-tls-certs") pod "model-serving-api-54d89bd5fc-9swdn" (UID: "cd2ea4b2-0027-4cd7-8e29-e36db1367a36") : secret "model-serving-api-tls" not found Jul 13 21:38:20.472686 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.472664 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhl7q\" (UniqueName: \"kubernetes.io/projected/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-kube-api-access-lhl7q\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.565063 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kpgm\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-kube-api-access-8kpgm\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565243 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565074 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565243 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565243 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565243 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565151 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91425bbc-3037-4fb6-97e3-6bb657d0db32-local-certs\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565243 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-token\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565519 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-cacerts\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.565791 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.565736 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.567569 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.567542 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91425bbc-3037-4fb6-97e3-6bb657d0db32-local-certs\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.567645 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.567619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-cacerts\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.567681 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.567665 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.567714 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.567688 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.572709 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.572690 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kpgm\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-kube-api-access-8kpgm\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.573183 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.573148 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-token\") pod \"istiod-openshift-gateway-7574459656-pcdbs\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.716331 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.716298 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:20.882081 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.882048 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs"] Jul 13 21:38:20.884893 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:20.884862 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91425bbc_3037_4fb6_97e3_6bb657d0db32.slice/crio-61cbb198d969f3ec35e1ed7b9c5ce21b1e062caf7243614691e9e63f4e3c4fc0 WatchSource:0}: Error finding container 61cbb198d969f3ec35e1ed7b9c5ce21b1e062caf7243614691e9e63f4e3c4fc0: Status 404 returned error can't find the container with id 61cbb198d969f3ec35e1ed7b9c5ce21b1e062caf7243614691e9e63f4e3c4fc0 Jul 13 21:38:20.968946 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.968885 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-tls-certs\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:20.971127 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:20.971107 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2ea4b2-0027-4cd7-8e29-e36db1367a36-tls-certs\") pod \"model-serving-api-54d89bd5fc-9swdn\" (UID: \"cd2ea4b2-0027-4cd7-8e29-e36db1367a36\") " pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:21.171557 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:21.171493 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" event={"ID":"91425bbc-3037-4fb6-97e3-6bb657d0db32","Type":"ContainerStarted","Data":"61cbb198d969f3ec35e1ed7b9c5ce21b1e062caf7243614691e9e63f4e3c4fc0"} Jul 13 21:38:21.172792 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:21.172753 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" event={"ID":"7d3a8120-670f-41a4-bfcb-1cce715a150b","Type":"ContainerStarted","Data":"5a06cd8654e15095bb4c77b043174a3c473554fd121b3ce1efe182266d0dd560"} Jul 13 21:38:21.194635 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:21.194607 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:21.371269 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:21.371240 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9swdn"] Jul 13 21:38:21.384578 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:21.377343 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2ea4b2_0027_4cd7_8e29_e36db1367a36.slice/crio-6dea178bfeeba60ea026df5e0d5ffc4c66a30516f1cfaff00c722c5195b46549 WatchSource:0}: Error finding container 6dea178bfeeba60ea026df5e0d5ffc4c66a30516f1cfaff00c722c5195b46549: Status 404 returned error can't find the container with id 6dea178bfeeba60ea026df5e0d5ffc4c66a30516f1cfaff00c722c5195b46549 Jul 13 21:38:22.180355 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:22.180279 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" event={"ID":"cd2ea4b2-0027-4cd7-8e29-e36db1367a36","Type":"ContainerStarted","Data":"6dea178bfeeba60ea026df5e0d5ffc4c66a30516f1cfaff00c722c5195b46549"} Jul 13 21:38:25.196835 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.196793 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" event={"ID":"cd2ea4b2-0027-4cd7-8e29-e36db1367a36","Type":"ContainerStarted","Data":"0dd463833ff4637a426fd0edfbcf1efe811ec640e2907881a425be2a4555babc"} Jul 13 21:38:25.197248 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.196971 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:25.198477 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.198445 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" event={"ID":"3d654a0c-921f-475d-b656-2795bd12952a","Type":"ContainerStarted","Data":"21c0b73e691052cf5707afd9b615b5fe001a9e1f6812c2bcc6b26a490cf9cbac"} Jul 13 21:38:25.200140 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.200119 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" event={"ID":"7d3a8120-670f-41a4-bfcb-1cce715a150b","Type":"ContainerStarted","Data":"3b9af9caba0850ef63af774f238089884cee3d1584c3c3c4e6a99d2b8446dc40"} Jul 13 21:38:25.200263 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.200175 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:25.213201 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.213161 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" podStartSLOduration=2.329009194 podStartE2EDuration="5.213145028s" totalImagesPullingTime="2.884135834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:25.212158184 +0000 UTC m=+526.031667994" watchObservedRunningTime="2026-07-13 21:38:25.213145028 +0000 UTC m=+526.032654841" Jul 13 21:38:25.226095 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.226055 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" podStartSLOduration=2.229976242 podStartE2EDuration="6.226042203s" totalImagesPullingTime="3.996065961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:25.225694449 +0000 UTC m=+526.045204282" watchObservedRunningTime="2026-07-13 21:38:25.226042203 +0000 UTC m=+526.045552029" Jul 13 21:38:25.239996 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:25.239964 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bff766844-kh9fs" podStartSLOduration=2.312428356 podStartE2EDuration="7.239951632s" totalImagesPullingTime="4.927523276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:25.239623122 +0000 UTC m=+526.059132956" watchObservedRunningTime="2026-07-13 21:38:25.239951632 +0000 UTC m=+526.059461422" Jul 13 21:38:29.779464 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.779436 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb"] Jul 13 21:38:29.783194 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.783171 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.786926 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.786907 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 21:38:29.787575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.787555 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cbxpx\"" Jul 13 21:38:29.794814 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.794794 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p"] Jul 13 21:38:29.798173 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.798152 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:29.800154 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.800132 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb"] Jul 13 21:38:29.801571 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.801553 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zpn22\"" Jul 13 21:38:29.802103 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.802084 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 21:38:29.815471 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.815436 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p"] Jul 13 21:38:29.830403 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.830372 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:38:29.830482 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.830447 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:38:29.842941 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.842920 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f41427fd-dc84-4aec-aeb7-2a70a840f081-cert\") pod \"kserve-controller-manager-d6f4b65b8-ptxzb\" (UID: \"f41427fd-dc84-4aec-aeb7-2a70a840f081\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.843040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.842967 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcjbn\" (UniqueName: \"kubernetes.io/projected/a108cde0-a23b-46fc-81a7-85d22163d74e-kube-api-access-tcjbn\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:29.843040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.843000 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a108cde0-a23b-46fc-81a7-85d22163d74e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:29.843040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.843023 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjl7\" (UniqueName: \"kubernetes.io/projected/f41427fd-dc84-4aec-aeb7-2a70a840f081-kube-api-access-kwjl7\") pod \"kserve-controller-manager-d6f4b65b8-ptxzb\" (UID: \"f41427fd-dc84-4aec-aeb7-2a70a840f081\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.943517 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.943481 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjl7\" (UniqueName: \"kubernetes.io/projected/f41427fd-dc84-4aec-aeb7-2a70a840f081-kube-api-access-kwjl7\") pod \"kserve-controller-manager-d6f4b65b8-ptxzb\" (UID: \"f41427fd-dc84-4aec-aeb7-2a70a840f081\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.943685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.943535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f41427fd-dc84-4aec-aeb7-2a70a840f081-cert\") pod \"kserve-controller-manager-d6f4b65b8-ptxzb\" (UID: \"f41427fd-dc84-4aec-aeb7-2a70a840f081\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.943685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.943571 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcjbn\" (UniqueName: \"kubernetes.io/projected/a108cde0-a23b-46fc-81a7-85d22163d74e-kube-api-access-tcjbn\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:29.943685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.943593 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a108cde0-a23b-46fc-81a7-85d22163d74e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:29.943685 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:29.943679 2583 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 13 21:38:29.943918 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:29.943724 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a108cde0-a23b-46fc-81a7-85d22163d74e-cert podName:a108cde0-a23b-46fc-81a7-85d22163d74e nodeName:}" failed. No retries permitted until 2026-07-13 21:38:30.443708891 +0000 UTC m=+531.263218678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a108cde0-a23b-46fc-81a7-85d22163d74e-cert") pod "llmisvc-controller-manager-777c6bd7d5-hjj6p" (UID: "a108cde0-a23b-46fc-81a7-85d22163d74e") : secret "llmisvc-webhook-server-cert" not found Jul 13 21:38:29.946076 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.946051 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f41427fd-dc84-4aec-aeb7-2a70a840f081-cert\") pod \"kserve-controller-manager-d6f4b65b8-ptxzb\" (UID: \"f41427fd-dc84-4aec-aeb7-2a70a840f081\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.957145 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.957121 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjl7\" (UniqueName: \"kubernetes.io/projected/f41427fd-dc84-4aec-aeb7-2a70a840f081-kube-api-access-kwjl7\") pod \"kserve-controller-manager-d6f4b65b8-ptxzb\" (UID: \"f41427fd-dc84-4aec-aeb7-2a70a840f081\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:29.967155 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:29.967131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcjbn\" (UniqueName: \"kubernetes.io/projected/a108cde0-a23b-46fc-81a7-85d22163d74e-kube-api-access-tcjbn\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:30.095824 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.095730 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:30.224822 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.224792 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb"] Jul 13 21:38:30.227786 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:30.227749 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf41427fd_dc84_4aec_aeb7_2a70a840f081.slice/crio-722c384f2b47d51b7e9e4f6a66a1df3927b93bca4348f60e044a88d3a4fbb222 WatchSource:0}: Error finding container 722c384f2b47d51b7e9e4f6a66a1df3927b93bca4348f60e044a88d3a4fbb222: Status 404 returned error can't find the container with id 722c384f2b47d51b7e9e4f6a66a1df3927b93bca4348f60e044a88d3a4fbb222 Jul 13 21:38:30.228907 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.228879 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" event={"ID":"91425bbc-3037-4fb6-97e3-6bb657d0db32","Type":"ContainerStarted","Data":"03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82"} Jul 13 21:38:30.228974 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.228937 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:30.230224 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.230200 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pcdbs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 21:38:30.230294 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.230252 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 21:38:30.255917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.255878 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" podStartSLOduration=1.313430286 podStartE2EDuration="10.255863606s" totalImagesPullingTime="8.94243332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:30.254213542 +0000 UTC m=+531.073723410" watchObservedRunningTime="2026-07-13 21:38:30.255863606 +0000 UTC m=+531.075373414" Jul 13 21:38:30.448479 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.448454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a108cde0-a23b-46fc-81a7-85d22163d74e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:30.450666 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.450638 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a108cde0-a23b-46fc-81a7-85d22163d74e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hjj6p\" (UID: \"a108cde0-a23b-46fc-81a7-85d22163d74e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:30.706995 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.706922 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:30.836242 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:30.836213 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p"] Jul 13 21:38:30.837387 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:30.837361 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda108cde0_a23b_46fc_81a7_85d22163d74e.slice/crio-7eeb630fbd6d4bf728674b194013e0f7f29158b1f7768c0fced281b9556ca0ad WatchSource:0}: Error finding container 7eeb630fbd6d4bf728674b194013e0f7f29158b1f7768c0fced281b9556ca0ad: Status 404 returned error can't find the container with id 7eeb630fbd6d4bf728674b194013e0f7f29158b1f7768c0fced281b9556ca0ad Jul 13 21:38:31.236796 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:31.236731 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" event={"ID":"a108cde0-a23b-46fc-81a7-85d22163d74e","Type":"ContainerStarted","Data":"7eeb630fbd6d4bf728674b194013e0f7f29158b1f7768c0fced281b9556ca0ad"} Jul 13 21:38:31.238315 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:31.238285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" event={"ID":"f41427fd-dc84-4aec-aeb7-2a70a840f081","Type":"ContainerStarted","Data":"722c384f2b47d51b7e9e4f6a66a1df3927b93bca4348f60e044a88d3a4fbb222"} Jul 13 21:38:31.241166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:31.240756 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pcdbs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 21:38:31.241166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:31.240821 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 21:38:32.148862 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.148587 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l"] Jul 13 21:38:32.153100 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.153074 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.156125 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.156104 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ztz5n\"" Jul 13 21:38:32.171556 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.171497 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l"] Jul 13 21:38:32.264425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264507 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95af6daa-68a4-4861-95d7-40c4ee1c761a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264541 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264568 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264962 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264620 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzx77\" (UniqueName: \"kubernetes.io/projected/95af6daa-68a4-4861-95d7-40c4ee1c761a-kube-api-access-wzx77\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264962 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264656 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.264962 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.264719 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365548 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365611 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95af6daa-68a4-4861-95d7-40c4ee1c761a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzx77\" (UniqueName: \"kubernetes.io/projected/95af6daa-68a4-4861-95d7-40c4ee1c761a-kube-api-access-wzx77\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.365816 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.365798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.366175 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.366034 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.366340 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.366295 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.366663 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.366643 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.367141 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.367116 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95af6daa-68a4-4861-95d7-40c4ee1c761a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.367237 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.367216 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.368529 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.368494 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.368649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.368624 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.376098 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.376055 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95af6daa-68a4-4861-95d7-40c4ee1c761a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.376813 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.376756 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzx77\" (UniqueName: \"kubernetes.io/projected/95af6daa-68a4-4861-95d7-40c4ee1c761a-kube-api-access-wzx77\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4npv5l\" (UID: \"95af6daa-68a4-4861-95d7-40c4ee1c761a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:32.467112 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:32.467077 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:33.027118 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:33.027094 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l"] Jul 13 21:38:33.029720 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:38:33.029676 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95af6daa_68a4_4861_95d7_40c4ee1c761a.slice/crio-61f4a963743653d24efa315e7b17562d1b1f62463a02cfa13b773dae2647a689 WatchSource:0}: Error finding container 61f4a963743653d24efa315e7b17562d1b1f62463a02cfa13b773dae2647a689: Status 404 returned error can't find the container with id 61f4a963743653d24efa315e7b17562d1b1f62463a02cfa13b773dae2647a689 Jul 13 21:38:33.250093 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:33.250054 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" event={"ID":"f41427fd-dc84-4aec-aeb7-2a70a840f081","Type":"ContainerStarted","Data":"55f69bc564c91910e8a58768b98fb4eb8b744d0269bb2edba361da97e0f8e349"} Jul 13 21:38:33.250517 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:33.250141 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:38:33.251271 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:33.251249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" event={"ID":"95af6daa-68a4-4861-95d7-40c4ee1c761a","Type":"ContainerStarted","Data":"61f4a963743653d24efa315e7b17562d1b1f62463a02cfa13b773dae2647a689"} Jul 13 21:38:33.267647 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:33.267597 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" podStartSLOduration=1.6045872490000002 podStartE2EDuration="4.267585064s" totalImagesPullingTime="2.662997815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:33.265825397 +0000 UTC m=+534.085335220" watchObservedRunningTime="2026-07-13 21:38:33.267585064 +0000 UTC m=+534.087094872" Jul 13 21:38:34.240051 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:34.240022 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:35.264943 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.264905 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" event={"ID":"a108cde0-a23b-46fc-81a7-85d22163d74e","Type":"ContainerStarted","Data":"0c91b6495abcc15f9beb452812bf08d3bfb608cc01b91b9aa43dc17469c6ed08"} Jul 13 21:38:35.265374 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.265014 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:38:35.282760 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.282707 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" podStartSLOduration=2.6654798619999998 podStartE2EDuration="6.282687464s" totalImagesPullingTime="3.617207602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:35.281424063 +0000 UTC m=+536.100933894" watchObservedRunningTime="2026-07-13 21:38:35.282687464 +0000 UTC m=+536.102197273" Jul 13 21:38:35.319463 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.319435 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs"] Jul 13 21:38:35.319662 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.319639 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerName="discovery" containerID="cri-o://03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82" gracePeriod=30 Jul 13 21:38:35.806728 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.806685 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:38:35.806829 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.806758 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:38:35.806829 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.806807 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:38:35.880665 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.880646 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:35.999785 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999688 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-dns-cert\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:35.999785 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999734 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-kubeconfig\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:35.999785 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999761 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-cacerts\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:36.000039 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999798 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kpgm\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-kube-api-access-8kpgm\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:36.000039 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999864 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-ca-configmap\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:36.000039 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999911 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91425bbc-3037-4fb6-97e3-6bb657d0db32-local-certs\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:36.000039 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:35.999958 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-token\") pod \"91425bbc-3037-4fb6-97e3-6bb657d0db32\" (UID: \"91425bbc-3037-4fb6-97e3-6bb657d0db32\") " Jul 13 21:38:36.000473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.000415 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:38:36.002537 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.002504 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:38:36.002898 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.002869 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:38:36.003094 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.003065 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-kube-api-access-8kpgm" (OuterVolumeSpecName: "kube-api-access-8kpgm") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "kube-api-access-8kpgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:38:36.003633 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.003601 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-token" (OuterVolumeSpecName: "istio-token") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:38:36.003633 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.003626 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-cacerts" (OuterVolumeSpecName: "cacerts") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:38:36.004175 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.004147 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91425bbc-3037-4fb6-97e3-6bb657d0db32-local-certs" (OuterVolumeSpecName: "local-certs") pod "91425bbc-3037-4fb6-97e3-6bb657d0db32" (UID: "91425bbc-3037-4fb6-97e3-6bb657d0db32"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:38:36.100542 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100519 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-ca-configmap\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.100542 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100541 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91425bbc-3037-4fb6-97e3-6bb657d0db32-local-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.100682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100552 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.100682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100561 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-csr-dns-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.100682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100569 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-istio-kubeconfig\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.100682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100578 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91425bbc-3037-4fb6-97e3-6bb657d0db32-cacerts\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.100682 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.100587 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kpgm\" (UniqueName: \"kubernetes.io/projected/91425bbc-3037-4fb6-97e3-6bb657d0db32-kube-api-access-8kpgm\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:38:36.207754 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.207731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-57hdq" Jul 13 21:38:36.209821 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.209799 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9swdn" Jul 13 21:38:36.271170 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.271096 2583 generic.go:356] "Generic (PLEG): container finished" podID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerID="03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82" exitCode=0 Jul 13 21:38:36.271170 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.271140 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" Jul 13 21:38:36.271593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.271187 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" event={"ID":"91425bbc-3037-4fb6-97e3-6bb657d0db32","Type":"ContainerDied","Data":"03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82"} Jul 13 21:38:36.271593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.271246 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs" event={"ID":"91425bbc-3037-4fb6-97e3-6bb657d0db32","Type":"ContainerDied","Data":"61cbb198d969f3ec35e1ed7b9c5ce21b1e062caf7243614691e9e63f4e3c4fc0"} Jul 13 21:38:36.271880 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.271855 2583 scope.go:117] "RemoveContainer" containerID="03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82" Jul 13 21:38:36.273891 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.273866 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" event={"ID":"95af6daa-68a4-4861-95d7-40c4ee1c761a","Type":"ContainerStarted","Data":"3c35eef98190eecfcaeb96de79d89e0a4d05414d4b26f6701c691c543f819a4b"} Jul 13 21:38:36.284655 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.284636 2583 scope.go:117] "RemoveContainer" containerID="03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82" Jul 13 21:38:36.284993 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:38:36.284967 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82\": container with ID starting with 03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82 not found: ID does not exist" containerID="03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82" Jul 13 21:38:36.285091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.285003 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82"} err="failed to get container status \"03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82\": rpc error: code = NotFound desc = could not find container \"03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82\": container with ID starting with 03521957c680c8541a73689a8c712c2d668ac989fae09313766eee802a4aab82 not found: ID does not exist" Jul 13 21:38:36.303049 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.303005 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" podStartSLOduration=1.528604396 podStartE2EDuration="4.302986901s" totalImagesPullingTime="2.774382505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:36.301540202 +0000 UTC m=+537.121050011" watchObservedRunningTime="2026-07-13 21:38:36.302986901 +0000 UTC m=+537.122496714" Jul 13 21:38:36.315636 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.315615 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs"] Jul 13 21:38:36.319765 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.319740 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pcdbs"] Jul 13 21:38:36.352305 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.352282 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" path="/var/lib/kubelet/pods/91425bbc-3037-4fb6-97e3-6bb657d0db32/volumes" Jul 13 21:38:36.467928 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.467900 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:36.471851 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:36.471832 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:37.280956 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:37.280916 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:38:37.281956 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:38:37.281934 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4npv5l" Jul 13 21:39:04.262685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:04.262653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ptxzb" Jul 13 21:39:06.276066 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:06.276033 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hjj6p" Jul 13 21:39:19.809627 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.809596 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dv7ph"] Jul 13 21:39:19.810073 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.809912 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerName="discovery" Jul 13 21:39:19.810073 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.809923 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerName="discovery" Jul 13 21:39:19.810073 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.809974 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="91425bbc-3037-4fb6-97e3-6bb657d0db32" containerName="discovery" Jul 13 21:39:19.816591 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.816568 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:19.819312 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.819218 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 21:39:19.820119 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.820095 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 21:39:19.820250 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.820224 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-crkc6\"" Jul 13 21:39:19.822458 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.822434 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dv7ph"] Jul 13 21:39:19.905303 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:19.905274 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp89m\" (UniqueName: \"kubernetes.io/projected/05bf8a78-55c4-40ba-bee0-c8a2278275df-kube-api-access-qp89m\") pod \"kuadrant-operator-catalog-dv7ph\" (UID: \"05bf8a78-55c4-40ba-bee0-c8a2278275df\") " pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:20.005913 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.005882 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp89m\" (UniqueName: \"kubernetes.io/projected/05bf8a78-55c4-40ba-bee0-c8a2278275df-kube-api-access-qp89m\") pod \"kuadrant-operator-catalog-dv7ph\" (UID: \"05bf8a78-55c4-40ba-bee0-c8a2278275df\") " pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:20.013780 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.013754 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp89m\" (UniqueName: \"kubernetes.io/projected/05bf8a78-55c4-40ba-bee0-c8a2278275df-kube-api-access-qp89m\") pod \"kuadrant-operator-catalog-dv7ph\" (UID: \"05bf8a78-55c4-40ba-bee0-c8a2278275df\") " pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:20.128296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.128243 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:20.163370 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.163341 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dv7ph"] Jul 13 21:39:20.252325 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.252303 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dv7ph"] Jul 13 21:39:20.254988 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:39:20.254963 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bf8a78_55c4_40ba_bee0_c8a2278275df.slice/crio-937b332c8534115dec31eaa577d27aa4d205ee6fcef9bd89d3eb622aefee13e9 WatchSource:0}: Error finding container 937b332c8534115dec31eaa577d27aa4d205ee6fcef9bd89d3eb622aefee13e9: Status 404 returned error can't find the container with id 937b332c8534115dec31eaa577d27aa4d205ee6fcef9bd89d3eb622aefee13e9 Jul 13 21:39:20.373505 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.373479 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t99mn"] Jul 13 21:39:20.376018 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.376004 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:20.383384 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.383363 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t99mn"] Jul 13 21:39:20.477128 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.477100 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" event={"ID":"05bf8a78-55c4-40ba-bee0-c8a2278275df","Type":"ContainerStarted","Data":"937b332c8534115dec31eaa577d27aa4d205ee6fcef9bd89d3eb622aefee13e9"} Jul 13 21:39:20.509455 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.509432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mljr7\" (UniqueName: \"kubernetes.io/projected/3f71e7f6-29b6-44b2-b2a2-18332f105064-kube-api-access-mljr7\") pod \"kuadrant-operator-catalog-t99mn\" (UID: \"3f71e7f6-29b6-44b2-b2a2-18332f105064\") " pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:20.610859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.610835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mljr7\" (UniqueName: \"kubernetes.io/projected/3f71e7f6-29b6-44b2-b2a2-18332f105064-kube-api-access-mljr7\") pod \"kuadrant-operator-catalog-t99mn\" (UID: \"3f71e7f6-29b6-44b2-b2a2-18332f105064\") " pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:20.619991 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.619972 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mljr7\" (UniqueName: \"kubernetes.io/projected/3f71e7f6-29b6-44b2-b2a2-18332f105064-kube-api-access-mljr7\") pod \"kuadrant-operator-catalog-t99mn\" (UID: \"3f71e7f6-29b6-44b2-b2a2-18332f105064\") " pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:20.686032 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.685999 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:20.807615 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:20.807587 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t99mn"] Jul 13 21:39:20.811206 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:39:20.811174 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f71e7f6_29b6_44b2_b2a2_18332f105064.slice/crio-189d04f5b6e2c6cf216b3e01aa4dc9d45b3009ff53baf44842ec156f5190dc79 WatchSource:0}: Error finding container 189d04f5b6e2c6cf216b3e01aa4dc9d45b3009ff53baf44842ec156f5190dc79: Status 404 returned error can't find the container with id 189d04f5b6e2c6cf216b3e01aa4dc9d45b3009ff53baf44842ec156f5190dc79 Jul 13 21:39:21.484347 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:21.484307 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" event={"ID":"3f71e7f6-29b6-44b2-b2a2-18332f105064","Type":"ContainerStarted","Data":"189d04f5b6e2c6cf216b3e01aa4dc9d45b3009ff53baf44842ec156f5190dc79"} Jul 13 21:39:22.496111 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.496072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" event={"ID":"3f71e7f6-29b6-44b2-b2a2-18332f105064","Type":"ContainerStarted","Data":"302f57462ec13a61ea58977ec58b803a36bc2444bbf1f3853b9e26ce9def68f6"} Jul 13 21:39:22.497647 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.497617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" event={"ID":"05bf8a78-55c4-40ba-bee0-c8a2278275df","Type":"ContainerStarted","Data":"dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe"} Jul 13 21:39:22.497925 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.497669 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" podUID="05bf8a78-55c4-40ba-bee0-c8a2278275df" containerName="registry-server" containerID="cri-o://dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe" gracePeriod=2 Jul 13 21:39:22.512039 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.511998 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" podStartSLOduration=0.981118607 podStartE2EDuration="2.511984766s" totalImagesPullingTime="1.530866159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:39:22.511052632 +0000 UTC m=+583.330562442" watchObservedRunningTime="2026-07-13 21:39:22.511984766 +0000 UTC m=+583.331494553" Jul 13 21:39:22.525983 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.525944 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" podStartSLOduration=1.442466007 podStartE2EDuration="3.525929939s" totalImagesPullingTime="2.083463932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:39:22.524811667 +0000 UTC m=+583.344321511" watchObservedRunningTime="2026-07-13 21:39:22.525929939 +0000 UTC m=+583.345439747" Jul 13 21:39:22.738362 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.738342 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dv7ph_05bf8a78-55c4-40ba-bee0-c8a2278275df/registry-server/0.log" Jul 13 21:39:22.740685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.740666 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:22.831463 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.831430 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp89m\" (UniqueName: \"kubernetes.io/projected/05bf8a78-55c4-40ba-bee0-c8a2278275df-kube-api-access-qp89m\") pod \"05bf8a78-55c4-40ba-bee0-c8a2278275df\" (UID: \"05bf8a78-55c4-40ba-bee0-c8a2278275df\") " Jul 13 21:39:22.833498 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.833476 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bf8a78-55c4-40ba-bee0-c8a2278275df-kube-api-access-qp89m" (OuterVolumeSpecName: "kube-api-access-qp89m") pod "05bf8a78-55c4-40ba-bee0-c8a2278275df" (UID: "05bf8a78-55c4-40ba-bee0-c8a2278275df"). InnerVolumeSpecName "kube-api-access-qp89m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:39:22.932285 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:22.932262 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp89m\" (UniqueName: \"kubernetes.io/projected/05bf8a78-55c4-40ba-bee0-c8a2278275df-kube-api-access-qp89m\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:23.503218 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.503192 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dv7ph_05bf8a78-55c4-40ba-bee0-c8a2278275df/registry-server/0.log" Jul 13 21:39:23.503651 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.503231 2583 generic.go:356] "Generic (PLEG): container finished" podID="05bf8a78-55c4-40ba-bee0-c8a2278275df" containerID="dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe" exitCode=1 Jul 13 21:39:23.503651 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.503296 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" Jul 13 21:39:23.503651 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.503320 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" event={"ID":"05bf8a78-55c4-40ba-bee0-c8a2278275df","Type":"ContainerDied","Data":"dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe"} Jul 13 21:39:23.503651 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.503367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dv7ph" event={"ID":"05bf8a78-55c4-40ba-bee0-c8a2278275df","Type":"ContainerDied","Data":"937b332c8534115dec31eaa577d27aa4d205ee6fcef9bd89d3eb622aefee13e9"} Jul 13 21:39:23.503651 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.503387 2583 scope.go:117] "RemoveContainer" containerID="dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe" Jul 13 21:39:23.513237 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.513217 2583 scope.go:117] "RemoveContainer" containerID="dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe" Jul 13 21:39:23.513484 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:39:23.513466 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe\": container with ID starting with dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe not found: ID does not exist" containerID="dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe" Jul 13 21:39:23.513523 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.513493 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe"} err="failed to get container status \"dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe\": rpc error: code = NotFound desc = could not find container \"dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe\": container with ID starting with dfddd642ca8a9803bc57de597d792661416ac6d928216b72397fefbb352c7ebe not found: ID does not exist" Jul 13 21:39:23.526081 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.526062 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dv7ph"] Jul 13 21:39:23.528809 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:23.528789 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dv7ph"] Jul 13 21:39:24.352122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:24.352087 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05bf8a78-55c4-40ba-bee0-c8a2278275df" path="/var/lib/kubelet/pods/05bf8a78-55c4-40ba-bee0-c8a2278275df/volumes" Jul 13 21:39:30.686695 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:30.686653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:30.687191 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:30.686719 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:30.708657 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:30.708637 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:31.560475 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:31.560447 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-t99mn" Jul 13 21:39:40.412000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:40.411903 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:39:40.433865 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:40.412753 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:39:49.894731 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.894701 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv"] Jul 13 21:39:49.895122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.895036 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05bf8a78-55c4-40ba-bee0-c8a2278275df" containerName="registry-server" Jul 13 21:39:49.895122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.895051 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05bf8a78-55c4-40ba-bee0-c8a2278275df" containerName="registry-server" Jul 13 21:39:49.895122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.895108 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05bf8a78-55c4-40ba-bee0-c8a2278275df" containerName="registry-server" Jul 13 21:39:49.898256 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.898237 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:49.900556 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.900535 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mw5h6\"" Jul 13 21:39:49.906882 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.906861 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv"] Jul 13 21:39:49.922420 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.922390 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b57x8\" (UniqueName: \"kubernetes.io/projected/56237d87-337e-4ac6-8202-fd631a54c130-kube-api-access-b57x8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:49.922526 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.922432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:49.922586 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.922523 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:49.993566 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.993546 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw"] Jul 13 21:39:49.996923 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:49.996906 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.005036 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.005016 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw"] Jul 13 21:39:50.023362 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b57x8\" (UniqueName: \"kubernetes.io/projected/56237d87-337e-4ac6-8202-fd631a54c130-kube-api-access-b57x8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.023455 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023375 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.023455 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023404 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.023540 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.023603 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023585 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.023673 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr96\" (UniqueName: \"kubernetes.io/projected/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-kube-api-access-jgr96\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.023761 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023742 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.023821 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.023754 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.031023 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.031003 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b57x8\" (UniqueName: \"kubernetes.io/projected/56237d87-337e-4ac6-8202-fd631a54c130-kube-api-access-b57x8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.124733 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.124703 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr96\" (UniqueName: \"kubernetes.io/projected/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-kube-api-access-jgr96\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.124901 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.124755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.124901 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.124830 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.125133 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.125115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.125182 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.125168 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.132648 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.132622 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr96\" (UniqueName: \"kubernetes.io/projected/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-kube-api-access-jgr96\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.207044 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.207015 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:50.292498 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.292471 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l"] Jul 13 21:39:50.297004 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.296979 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.305459 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.305058 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l"] Jul 13 21:39:50.305915 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.305688 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:50.325767 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.325728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.325902 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.325884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.325973 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.325910 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24pn9\" (UniqueName: \"kubernetes.io/projected/f8fe055a-6f53-408a-8d90-cfad5acc4780-kube-api-access-24pn9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.336161 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.336144 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv"] Jul 13 21:39:50.338412 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:39:50.338382 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56237d87_337e_4ac6_8202_fd631a54c130.slice/crio-b7e0b88a04f57f998080148b20f52d477c10a80e0cebd35d3dad3474fd826c76 WatchSource:0}: Error finding container b7e0b88a04f57f998080148b20f52d477c10a80e0cebd35d3dad3474fd826c76: Status 404 returned error can't find the container with id b7e0b88a04f57f998080148b20f52d477c10a80e0cebd35d3dad3474fd826c76 Jul 13 21:39:50.426717 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.426690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.426717 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.426721 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24pn9\" (UniqueName: \"kubernetes.io/projected/f8fe055a-6f53-408a-8d90-cfad5acc4780-kube-api-access-24pn9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.426909 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.426867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.427102 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.427081 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.427182 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.427160 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.434650 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.434628 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24pn9\" (UniqueName: \"kubernetes.io/projected/f8fe055a-6f53-408a-8d90-cfad5acc4780-kube-api-access-24pn9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.444916 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.444821 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw"] Jul 13 21:39:50.489626 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:39:50.489555 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a06cd3_d24b_4a2c_b4b0_0d38d966bc73.slice/crio-40d1f691c3616d971735fcc6a80f2fcbed743f3e6debc5e61db0996564af0991 WatchSource:0}: Error finding container 40d1f691c3616d971735fcc6a80f2fcbed743f3e6debc5e61db0996564af0991: Status 404 returned error can't find the container with id 40d1f691c3616d971735fcc6a80f2fcbed743f3e6debc5e61db0996564af0991 Jul 13 21:39:50.607402 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.607376 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:50.624945 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.624921 2583 generic.go:356] "Generic (PLEG): container finished" podID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerID="e6bb62123b3f0c5bb5c46a7ca6d94f5e2aa1bde0dda31970a12bd9e2fce9c7d7" exitCode=0 Jul 13 21:39:50.625053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.625007 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" event={"ID":"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73","Type":"ContainerDied","Data":"e6bb62123b3f0c5bb5c46a7ca6d94f5e2aa1bde0dda31970a12bd9e2fce9c7d7"} Jul 13 21:39:50.625116 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.625061 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" event={"ID":"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73","Type":"ContainerStarted","Data":"40d1f691c3616d971735fcc6a80f2fcbed743f3e6debc5e61db0996564af0991"} Jul 13 21:39:50.626636 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.626615 2583 generic.go:356] "Generic (PLEG): container finished" podID="56237d87-337e-4ac6-8202-fd631a54c130" containerID="d4fa391ca8f6165dcd9347cf338ddf088193b5fe22de47d1f6e51cc159bb59ff" exitCode=0 Jul 13 21:39:50.626732 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.626675 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" event={"ID":"56237d87-337e-4ac6-8202-fd631a54c130","Type":"ContainerDied","Data":"d4fa391ca8f6165dcd9347cf338ddf088193b5fe22de47d1f6e51cc159bb59ff"} Jul 13 21:39:50.626732 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.626713 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" event={"ID":"56237d87-337e-4ac6-8202-fd631a54c130","Type":"ContainerStarted","Data":"b7e0b88a04f57f998080148b20f52d477c10a80e0cebd35d3dad3474fd826c76"} Jul 13 21:39:50.696943 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.696919 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m"] Jul 13 21:39:50.701552 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.701534 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.708509 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.708490 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m"] Jul 13 21:39:50.729247 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.729225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.729351 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.729299 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnqkl\" (UniqueName: \"kubernetes.io/projected/07b95b97-1c92-43b0-adf3-700bf6dc7c22-kube-api-access-bnqkl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.729400 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.729388 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.737912 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.737894 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l"] Jul 13 21:39:50.739038 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:39:50.739015 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fe055a_6f53_408a_8d90_cfad5acc4780.slice/crio-8ee355fee31aced3b33a2470cf009f1b443f7ea384bce356e94261bed0299cf9 WatchSource:0}: Error finding container 8ee355fee31aced3b33a2470cf009f1b443f7ea384bce356e94261bed0299cf9: Status 404 returned error can't find the container with id 8ee355fee31aced3b33a2470cf009f1b443f7ea384bce356e94261bed0299cf9 Jul 13 21:39:50.830439 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.830410 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.830622 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.830501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.830622 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.830542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnqkl\" (UniqueName: \"kubernetes.io/projected/07b95b97-1c92-43b0-adf3-700bf6dc7c22-kube-api-access-bnqkl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.830871 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.830849 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.830871 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.830860 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:50.838865 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:50.838841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnqkl\" (UniqueName: \"kubernetes.io/projected/07b95b97-1c92-43b0-adf3-700bf6dc7c22-kube-api-access-bnqkl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:51.014483 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.014417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:51.139736 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.139711 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m"] Jul 13 21:39:51.141927 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:39:51.141903 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b95b97_1c92_43b0_adf3_700bf6dc7c22.slice/crio-d85657aadd4536fd6908c764f7aec0eb37649a519ca6714f908eaf5969730a65 WatchSource:0}: Error finding container d85657aadd4536fd6908c764f7aec0eb37649a519ca6714f908eaf5969730a65: Status 404 returned error can't find the container with id d85657aadd4536fd6908c764f7aec0eb37649a519ca6714f908eaf5969730a65 Jul 13 21:39:51.632553 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.632528 2583 generic.go:356] "Generic (PLEG): container finished" podID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerID="ffdb5aca2abd2fb3cbdbd6477e207db7cdddd252f1e7725143a3d7afabbe20e4" exitCode=0 Jul 13 21:39:51.632656 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.632601 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" event={"ID":"07b95b97-1c92-43b0-adf3-700bf6dc7c22","Type":"ContainerDied","Data":"ffdb5aca2abd2fb3cbdbd6477e207db7cdddd252f1e7725143a3d7afabbe20e4"} Jul 13 21:39:51.632656 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.632647 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" event={"ID":"07b95b97-1c92-43b0-adf3-700bf6dc7c22","Type":"ContainerStarted","Data":"d85657aadd4536fd6908c764f7aec0eb37649a519ca6714f908eaf5969730a65"} Jul 13 21:39:51.634337 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.634314 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerID="87d20f3b31b774b292f001d96e9a439642d8f8d2aac5ecdd8c189f187d004666" exitCode=0 Jul 13 21:39:51.634493 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.634461 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" event={"ID":"f8fe055a-6f53-408a-8d90-cfad5acc4780","Type":"ContainerDied","Data":"87d20f3b31b774b292f001d96e9a439642d8f8d2aac5ecdd8c189f187d004666"} Jul 13 21:39:51.634594 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.634501 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" event={"ID":"f8fe055a-6f53-408a-8d90-cfad5acc4780","Type":"ContainerStarted","Data":"8ee355fee31aced3b33a2470cf009f1b443f7ea384bce356e94261bed0299cf9"} Jul 13 21:39:51.637213 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.637188 2583 generic.go:356] "Generic (PLEG): container finished" podID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerID="f1a20769e0a0878579dc10d2fc64efc88faa26827699af3b6523622d0995425f" exitCode=0 Jul 13 21:39:51.637294 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:51.637260 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" event={"ID":"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73","Type":"ContainerDied","Data":"f1a20769e0a0878579dc10d2fc64efc88faa26827699af3b6523622d0995425f"} Jul 13 21:39:52.643953 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.643928 2583 generic.go:356] "Generic (PLEG): container finished" podID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerID="5049bcef8dc18a4774b659f1d113010eb53cb80af8cb87e8f380ab1ff8dcd60f" exitCode=0 Jul 13 21:39:52.644262 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.644009 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" event={"ID":"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73","Type":"ContainerDied","Data":"5049bcef8dc18a4774b659f1d113010eb53cb80af8cb87e8f380ab1ff8dcd60f"} Jul 13 21:39:52.645749 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.645730 2583 generic.go:356] "Generic (PLEG): container finished" podID="56237d87-337e-4ac6-8202-fd631a54c130" containerID="a05bef3ed74be1761ab268cd7a4b20bc70eec909efab31e57242a0fcd8b05d78" exitCode=0 Jul 13 21:39:52.645854 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.645809 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" event={"ID":"56237d87-337e-4ac6-8202-fd631a54c130","Type":"ContainerDied","Data":"a05bef3ed74be1761ab268cd7a4b20bc70eec909efab31e57242a0fcd8b05d78"} Jul 13 21:39:52.647656 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.647637 2583 generic.go:356] "Generic (PLEG): container finished" podID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerID="9e7f3b44163a9ccbac1d479755e52a633ce8a8ac6e161ad3f7b31d9bb26cd01d" exitCode=0 Jul 13 21:39:52.647748 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.647666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" event={"ID":"07b95b97-1c92-43b0-adf3-700bf6dc7c22","Type":"ContainerDied","Data":"9e7f3b44163a9ccbac1d479755e52a633ce8a8ac6e161ad3f7b31d9bb26cd01d"} Jul 13 21:39:52.649556 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.649535 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerID="c628edffa65df5c272b8415f0b04404f13246d1ecc529dd6d42a199b7dbb9b6f" exitCode=0 Jul 13 21:39:52.649623 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:52.649586 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" event={"ID":"f8fe055a-6f53-408a-8d90-cfad5acc4780","Type":"ContainerDied","Data":"c628edffa65df5c272b8415f0b04404f13246d1ecc529dd6d42a199b7dbb9b6f"} Jul 13 21:39:53.656729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.656696 2583 generic.go:356] "Generic (PLEG): container finished" podID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerID="55f1f0939f0a9ecf4f25f186a8938e75a3148faf24a124628a401e54ac0e33a6" exitCode=0 Jul 13 21:39:53.657137 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.656794 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" event={"ID":"07b95b97-1c92-43b0-adf3-700bf6dc7c22","Type":"ContainerDied","Data":"55f1f0939f0a9ecf4f25f186a8938e75a3148faf24a124628a401e54ac0e33a6"} Jul 13 21:39:53.658680 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.658658 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerID="2898d4507837b710b7e437b14a3039447e984a982387f0fd841bc426ec506b0e" exitCode=0 Jul 13 21:39:53.658804 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.658732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" event={"ID":"f8fe055a-6f53-408a-8d90-cfad5acc4780","Type":"ContainerDied","Data":"2898d4507837b710b7e437b14a3039447e984a982387f0fd841bc426ec506b0e"} Jul 13 21:39:53.660867 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.660850 2583 generic.go:356] "Generic (PLEG): container finished" podID="56237d87-337e-4ac6-8202-fd631a54c130" containerID="d3dee7035b243fe8f4e274d982c9cca7f3d03efe7acba269c431e8b7410bcb5f" exitCode=0 Jul 13 21:39:53.660947 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.660882 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" event={"ID":"56237d87-337e-4ac6-8202-fd631a54c130","Type":"ContainerDied","Data":"d3dee7035b243fe8f4e274d982c9cca7f3d03efe7acba269c431e8b7410bcb5f"} Jul 13 21:39:53.785675 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.785652 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:53.850921 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.850891 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgr96\" (UniqueName: \"kubernetes.io/projected/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-kube-api-access-jgr96\") pod \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " Jul 13 21:39:53.851044 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.850943 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-bundle\") pod \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " Jul 13 21:39:53.851044 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.850966 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-util\") pod \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\" (UID: \"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73\") " Jul 13 21:39:53.851614 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.851581 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-bundle" (OuterVolumeSpecName: "bundle") pod "b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" (UID: "b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:53.852845 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.852822 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-kube-api-access-jgr96" (OuterVolumeSpecName: "kube-api-access-jgr96") pod "b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" (UID: "b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73"). InnerVolumeSpecName "kube-api-access-jgr96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:39:53.856389 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.856358 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-util" (OuterVolumeSpecName: "util") pod "b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" (UID: "b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:53.951990 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.951965 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgr96\" (UniqueName: \"kubernetes.io/projected/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-kube-api-access-jgr96\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:53.951990 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.951986 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:53.952159 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:53.951995 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:54.672570 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.672490 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" Jul 13 21:39:54.672570 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.672516 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw" event={"ID":"b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73","Type":"ContainerDied","Data":"40d1f691c3616d971735fcc6a80f2fcbed743f3e6debc5e61db0996564af0991"} Jul 13 21:39:54.673079 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.672589 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40d1f691c3616d971735fcc6a80f2fcbed743f3e6debc5e61db0996564af0991" Jul 13 21:39:54.819002 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.818978 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:54.857708 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.857688 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b57x8\" (UniqueName: \"kubernetes.io/projected/56237d87-337e-4ac6-8202-fd631a54c130-kube-api-access-b57x8\") pod \"56237d87-337e-4ac6-8202-fd631a54c130\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " Jul 13 21:39:54.857820 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.857750 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-util\") pod \"56237d87-337e-4ac6-8202-fd631a54c130\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " Jul 13 21:39:54.857889 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.857830 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-bundle\") pod \"56237d87-337e-4ac6-8202-fd631a54c130\" (UID: \"56237d87-337e-4ac6-8202-fd631a54c130\") " Jul 13 21:39:54.858159 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.858138 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:54.858427 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.858406 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-bundle" (OuterVolumeSpecName: "bundle") pod "56237d87-337e-4ac6-8202-fd631a54c130" (UID: "56237d87-337e-4ac6-8202-fd631a54c130"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:54.860017 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.859989 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56237d87-337e-4ac6-8202-fd631a54c130-kube-api-access-b57x8" (OuterVolumeSpecName: "kube-api-access-b57x8") pod "56237d87-337e-4ac6-8202-fd631a54c130" (UID: "56237d87-337e-4ac6-8202-fd631a54c130"). InnerVolumeSpecName "kube-api-access-b57x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:39:54.861592 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.861575 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:54.863948 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.863909 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-util" (OuterVolumeSpecName: "util") pod "56237d87-337e-4ac6-8202-fd631a54c130" (UID: "56237d87-337e-4ac6-8202-fd631a54c130"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:54.958845 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.958824 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-bundle\") pod \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " Jul 13 21:39:54.958977 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.958853 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-util\") pod \"f8fe055a-6f53-408a-8d90-cfad5acc4780\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " Jul 13 21:39:54.958977 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.958879 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnqkl\" (UniqueName: \"kubernetes.io/projected/07b95b97-1c92-43b0-adf3-700bf6dc7c22-kube-api-access-bnqkl\") pod \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " Jul 13 21:39:54.958977 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.958899 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24pn9\" (UniqueName: \"kubernetes.io/projected/f8fe055a-6f53-408a-8d90-cfad5acc4780-kube-api-access-24pn9\") pod \"f8fe055a-6f53-408a-8d90-cfad5acc4780\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " Jul 13 21:39:54.958977 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.958951 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-bundle\") pod \"f8fe055a-6f53-408a-8d90-cfad5acc4780\" (UID: \"f8fe055a-6f53-408a-8d90-cfad5acc4780\") " Jul 13 21:39:54.959187 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.958995 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-util\") pod \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\" (UID: \"07b95b97-1c92-43b0-adf3-700bf6dc7c22\") " Jul 13 21:39:54.959187 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.959161 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b57x8\" (UniqueName: \"kubernetes.io/projected/56237d87-337e-4ac6-8202-fd631a54c130-kube-api-access-b57x8\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:54.959187 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.959178 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:54.959339 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.959192 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56237d87-337e-4ac6-8202-fd631a54c130-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:54.959393 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.959372 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-bundle" (OuterVolumeSpecName: "bundle") pod "07b95b97-1c92-43b0-adf3-700bf6dc7c22" (UID: "07b95b97-1c92-43b0-adf3-700bf6dc7c22"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:54.959601 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.959577 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-bundle" (OuterVolumeSpecName: "bundle") pod "f8fe055a-6f53-408a-8d90-cfad5acc4780" (UID: "f8fe055a-6f53-408a-8d90-cfad5acc4780"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:54.961040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.961012 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fe055a-6f53-408a-8d90-cfad5acc4780-kube-api-access-24pn9" (OuterVolumeSpecName: "kube-api-access-24pn9") pod "f8fe055a-6f53-408a-8d90-cfad5acc4780" (UID: "f8fe055a-6f53-408a-8d90-cfad5acc4780"). InnerVolumeSpecName "kube-api-access-24pn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:39:54.961040 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.961030 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b95b97-1c92-43b0-adf3-700bf6dc7c22-kube-api-access-bnqkl" (OuterVolumeSpecName: "kube-api-access-bnqkl") pod "07b95b97-1c92-43b0-adf3-700bf6dc7c22" (UID: "07b95b97-1c92-43b0-adf3-700bf6dc7c22"). InnerVolumeSpecName "kube-api-access-bnqkl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:39:54.968307 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.968280 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-util" (OuterVolumeSpecName: "util") pod "f8fe055a-6f53-408a-8d90-cfad5acc4780" (UID: "f8fe055a-6f53-408a-8d90-cfad5acc4780"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:54.968636 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:54.968617 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-util" (OuterVolumeSpecName: "util") pod "07b95b97-1c92-43b0-adf3-700bf6dc7c22" (UID: "07b95b97-1c92-43b0-adf3-700bf6dc7c22"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:39:55.060271 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.060247 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:55.060271 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.060269 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:55.060377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.060277 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07b95b97-1c92-43b0-adf3-700bf6dc7c22-bundle\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:55.060377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.060284 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8fe055a-6f53-408a-8d90-cfad5acc4780-util\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:55.060377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.060292 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnqkl\" (UniqueName: \"kubernetes.io/projected/07b95b97-1c92-43b0-adf3-700bf6dc7c22-kube-api-access-bnqkl\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:55.060377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.060302 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24pn9\" (UniqueName: \"kubernetes.io/projected/f8fe055a-6f53-408a-8d90-cfad5acc4780-kube-api-access-24pn9\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:39:55.679994 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.679969 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" Jul 13 21:39:55.679994 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.679970 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m" event={"ID":"07b95b97-1c92-43b0-adf3-700bf6dc7c22","Type":"ContainerDied","Data":"d85657aadd4536fd6908c764f7aec0eb37649a519ca6714f908eaf5969730a65"} Jul 13 21:39:55.680456 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.680022 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d85657aadd4536fd6908c764f7aec0eb37649a519ca6714f908eaf5969730a65" Jul 13 21:39:55.682125 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.682101 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" Jul 13 21:39:55.682269 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.682132 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l" event={"ID":"f8fe055a-6f53-408a-8d90-cfad5acc4780","Type":"ContainerDied","Data":"8ee355fee31aced3b33a2470cf009f1b443f7ea384bce356e94261bed0299cf9"} Jul 13 21:39:55.682269 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.682172 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ee355fee31aced3b33a2470cf009f1b443f7ea384bce356e94261bed0299cf9" Jul 13 21:39:55.684183 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.684158 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" event={"ID":"56237d87-337e-4ac6-8202-fd631a54c130","Type":"ContainerDied","Data":"b7e0b88a04f57f998080148b20f52d477c10a80e0cebd35d3dad3474fd826c76"} Jul 13 21:39:55.684310 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.684189 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv" Jul 13 21:39:55.684310 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:39:55.684191 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7e0b88a04f57f998080148b20f52d477c10a80e0cebd35d3dad3474fd826c76" Jul 13 21:40:01.204921 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.204884 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5"] Jul 13 21:40:01.205345 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205315 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="extract" Jul 13 21:40:01.205345 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205332 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="extract" Jul 13 21:40:01.205345 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205346 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205354 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205365 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="extract" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205374 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="extract" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205389 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205397 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205411 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="util" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205419 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="util" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205429 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205437 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205448 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="util" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205456 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="util" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205475 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205484 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="pull" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205495 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="extract" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205504 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="extract" Jul 13 21:40:01.205520 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205521 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="extract" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205529 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="extract" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205537 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="util" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205545 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="util" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205555 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="util" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205565 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="util" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205641 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b95b97-1c92-43b0-adf3-700bf6dc7c22" containerName="extract" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205654 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73" containerName="extract" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205664 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fe055a-6f53-408a-8d90-cfad5acc4780" containerName="extract" Jul 13 21:40:01.206249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.205673 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="56237d87-337e-4ac6-8202-fd631a54c130" containerName="extract" Jul 13 21:40:01.212736 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.212715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:01.215151 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.215132 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7c82g\"" Jul 13 21:40:01.223412 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.223384 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5"] Jul 13 21:40:01.305750 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.305720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjf52\" (UniqueName: \"kubernetes.io/projected/387b0370-a3aa-495a-b6a6-36be6b94ba47-kube-api-access-kjf52\") pod \"authorino-operator-75cc8d5cb7-fs8q5\" (UID: \"387b0370-a3aa-495a-b6a6-36be6b94ba47\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:01.406127 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.406100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjf52\" (UniqueName: \"kubernetes.io/projected/387b0370-a3aa-495a-b6a6-36be6b94ba47-kube-api-access-kjf52\") pod \"authorino-operator-75cc8d5cb7-fs8q5\" (UID: \"387b0370-a3aa-495a-b6a6-36be6b94ba47\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:01.416949 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.416923 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjf52\" (UniqueName: \"kubernetes.io/projected/387b0370-a3aa-495a-b6a6-36be6b94ba47-kube-api-access-kjf52\") pod \"authorino-operator-75cc8d5cb7-fs8q5\" (UID: \"387b0370-a3aa-495a-b6a6-36be6b94ba47\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:01.522597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.522512 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:01.651532 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.651476 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5"] Jul 13 21:40:01.653934 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:40:01.653903 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387b0370_a3aa_495a_b6a6_36be6b94ba47.slice/crio-895b4857f1a7fc369cd885f3bca866095a0cfd6a50c69f9b9603f22f6858dd3a WatchSource:0}: Error finding container 895b4857f1a7fc369cd885f3bca866095a0cfd6a50c69f9b9603f22f6858dd3a: Status 404 returned error can't find the container with id 895b4857f1a7fc369cd885f3bca866095a0cfd6a50c69f9b9603f22f6858dd3a Jul 13 21:40:01.712476 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:01.712449 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" event={"ID":"387b0370-a3aa-495a-b6a6-36be6b94ba47","Type":"ContainerStarted","Data":"895b4857f1a7fc369cd885f3bca866095a0cfd6a50c69f9b9603f22f6858dd3a"} Jul 13 21:40:04.457015 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.456989 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88"] Jul 13 21:40:04.461768 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.461750 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:04.465119 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.465099 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mk5vt\"" Jul 13 21:40:04.472527 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.472508 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88"] Jul 13 21:40:04.530991 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.530964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw2gj\" (UniqueName: \"kubernetes.io/projected/d27ff499-3ca0-423b-b5a4-9e078c1ef589-kube-api-access-bw2gj\") pod \"limitador-operator-controller-manager-859d7b78bd-b7p88\" (UID: \"d27ff499-3ca0-423b-b5a4-9e078c1ef589\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:04.632362 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.632326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw2gj\" (UniqueName: \"kubernetes.io/projected/d27ff499-3ca0-423b-b5a4-9e078c1ef589-kube-api-access-bw2gj\") pod \"limitador-operator-controller-manager-859d7b78bd-b7p88\" (UID: \"d27ff499-3ca0-423b-b5a4-9e078c1ef589\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:04.663019 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.662997 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw2gj\" (UniqueName: \"kubernetes.io/projected/d27ff499-3ca0-423b-b5a4-9e078c1ef589-kube-api-access-bw2gj\") pod \"limitador-operator-controller-manager-859d7b78bd-b7p88\" (UID: \"d27ff499-3ca0-423b-b5a4-9e078c1ef589\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:04.729719 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.729649 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" event={"ID":"387b0370-a3aa-495a-b6a6-36be6b94ba47","Type":"ContainerStarted","Data":"084163c0e942306784538e5161e54a5b2f927f2a9d21ea68e20a8523f125a44a"} Jul 13 21:40:04.729860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.729820 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:04.771363 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.771318 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" podStartSLOduration=0.990841113 podStartE2EDuration="3.771303266s" totalImagesPullingTime="2.780462153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:04.769525359 +0000 UTC m=+625.589035167" watchObservedRunningTime="2026-07-13 21:40:04.771303266 +0000 UTC m=+625.590813140" Jul 13 21:40:04.801653 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.801626 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:04.951112 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:04.951089 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88"] Jul 13 21:40:04.952908 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:40:04.952882 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27ff499_3ca0_423b_b5a4_9e078c1ef589.slice/crio-645c8a425bfdfbc96b300c506f2304879b9228fc6541116d6331d4f323943753 WatchSource:0}: Error finding container 645c8a425bfdfbc96b300c506f2304879b9228fc6541116d6331d4f323943753: Status 404 returned error can't find the container with id 645c8a425bfdfbc96b300c506f2304879b9228fc6541116d6331d4f323943753 Jul 13 21:40:05.737356 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:05.737314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" event={"ID":"d27ff499-3ca0-423b-b5a4-9e078c1ef589","Type":"ContainerStarted","Data":"645c8a425bfdfbc96b300c506f2304879b9228fc6541116d6331d4f323943753"} Jul 13 21:40:06.743963 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:06.743937 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" event={"ID":"d27ff499-3ca0-423b-b5a4-9e078c1ef589","Type":"ContainerStarted","Data":"08c15622ecd85dbd64e680be5b2d68da576adf1f1e3e4f761b7c715f08f3d00d"} Jul 13 21:40:06.744278 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:06.744043 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:06.762601 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:06.762557 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" podStartSLOduration=1.086405712 podStartE2EDuration="2.762544177s" totalImagesPullingTime="1.676138465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:06.761316138 +0000 UTC m=+627.580825968" watchObservedRunningTime="2026-07-13 21:40:06.762544177 +0000 UTC m=+627.582053985" Jul 13 21:40:09.184504 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.184474 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb"] Jul 13 21:40:09.187971 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.187952 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.190684 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.190659 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v6jfl\"" Jul 13 21:40:09.211752 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.211733 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb"] Jul 13 21:40:09.271308 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.271280 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55700fca-27bd-4771-b611-7d0787a66ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.271405 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.271341 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbrlf\" (UniqueName: \"kubernetes.io/projected/55700fca-27bd-4771-b611-7d0787a66ae9-kube-api-access-gbrlf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.371985 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.371957 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbrlf\" (UniqueName: \"kubernetes.io/projected/55700fca-27bd-4771-b611-7d0787a66ae9-kube-api-access-gbrlf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.372091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.372021 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55700fca-27bd-4771-b611-7d0787a66ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.372335 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.372308 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55700fca-27bd-4771-b611-7d0787a66ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.380096 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.380067 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbrlf\" (UniqueName: \"kubernetes.io/projected/55700fca-27bd-4771-b611-7d0787a66ae9-kube-api-access-gbrlf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.498191 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.498134 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:09.631464 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.631434 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb"] Jul 13 21:40:09.635381 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:40:09.635354 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55700fca_27bd_4771_b611_7d0787a66ae9.slice/crio-63f6a341e3464d124b45e30460da11ed022a464ac23d7c62b80a9e16024fa98f WatchSource:0}: Error finding container 63f6a341e3464d124b45e30460da11ed022a464ac23d7c62b80a9e16024fa98f: Status 404 returned error can't find the container with id 63f6a341e3464d124b45e30460da11ed022a464ac23d7c62b80a9e16024fa98f Jul 13 21:40:09.765732 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:09.765655 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" event={"ID":"55700fca-27bd-4771-b611-7d0787a66ae9","Type":"ContainerStarted","Data":"63f6a341e3464d124b45e30460da11ed022a464ac23d7c62b80a9e16024fa98f"} Jul 13 21:40:15.740687 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:15.740656 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fs8q5" Jul 13 21:40:16.800378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:16.800341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" event={"ID":"55700fca-27bd-4771-b611-7d0787a66ae9","Type":"ContainerStarted","Data":"91c6114eb23ea37f7f52058669607958a98547340ab3032779e977a2af796e3a"} Jul 13 21:40:16.800737 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:16.800411 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:16.821387 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:16.821335 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" podStartSLOduration=0.799180055 podStartE2EDuration="7.82131976s" totalImagesPullingTime="7.022139705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:16.819698985 +0000 UTC m=+637.639208814" watchObservedRunningTime="2026-07-13 21:40:16.82131976 +0000 UTC m=+637.640829569" Jul 13 21:40:17.752028 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:17.751999 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:19.324309 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.324279 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw"] Jul 13 21:40:19.327824 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.327807 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.331143 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.331118 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 13 21:40:19.331316 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.331290 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mw5h6\"" Jul 13 21:40:19.331593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.331577 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 13 21:40:19.349931 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.349903 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw"] Jul 13 21:40:19.454934 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.454901 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02ef0294-96c7-45be-94d4-76d120d6e6b1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.455102 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.454943 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzhjx\" (UniqueName: \"kubernetes.io/projected/02ef0294-96c7-45be-94d4-76d120d6e6b1-kube-api-access-mzhjx\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.455102 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.454989 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02ef0294-96c7-45be-94d4-76d120d6e6b1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.556181 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.556146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02ef0294-96c7-45be-94d4-76d120d6e6b1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.556366 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.556219 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02ef0294-96c7-45be-94d4-76d120d6e6b1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.556366 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.556238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzhjx\" (UniqueName: \"kubernetes.io/projected/02ef0294-96c7-45be-94d4-76d120d6e6b1-kube-api-access-mzhjx\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.556366 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:19.556284 2583 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 13 21:40:19.556366 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:19.556356 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ef0294-96c7-45be-94d4-76d120d6e6b1-plugin-serving-cert podName:02ef0294-96c7-45be-94d4-76d120d6e6b1 nodeName:}" failed. No retries permitted until 2026-07-13 21:40:20.056339693 +0000 UTC m=+640.875849479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/02ef0294-96c7-45be-94d4-76d120d6e6b1-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-4s9mw" (UID: "02ef0294-96c7-45be-94d4-76d120d6e6b1") : secret "plugin-serving-cert" not found Jul 13 21:40:19.556992 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.556969 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02ef0294-96c7-45be-94d4-76d120d6e6b1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:19.565290 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:19.565270 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzhjx\" (UniqueName: \"kubernetes.io/projected/02ef0294-96c7-45be-94d4-76d120d6e6b1-kube-api-access-mzhjx\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:20.060911 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:20.060863 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02ef0294-96c7-45be-94d4-76d120d6e6b1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:20.063389 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:20.063355 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02ef0294-96c7-45be-94d4-76d120d6e6b1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4s9mw\" (UID: \"02ef0294-96c7-45be-94d4-76d120d6e6b1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:20.236277 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:20.236241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" Jul 13 21:40:20.371230 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:20.371200 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw"] Jul 13 21:40:20.372157 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:40:20.372133 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02ef0294_96c7_45be_94d4_76d120d6e6b1.slice/crio-f38550f09f1f55afddc78e0f986b2cd2f54968c6950272e6ce97e9b0abef33df WatchSource:0}: Error finding container f38550f09f1f55afddc78e0f986b2cd2f54968c6950272e6ce97e9b0abef33df: Status 404 returned error can't find the container with id f38550f09f1f55afddc78e0f986b2cd2f54968c6950272e6ce97e9b0abef33df Jul 13 21:40:20.821575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:20.821546 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" event={"ID":"02ef0294-96c7-45be-94d4-76d120d6e6b1","Type":"ContainerStarted","Data":"f38550f09f1f55afddc78e0f986b2cd2f54968c6950272e6ce97e9b0abef33df"} Jul 13 21:40:27.807897 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:27.807864 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:29.633021 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.632989 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb"] Jul 13 21:40:29.633445 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.633156 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" containerName="manager" containerID="cri-o://91c6114eb23ea37f7f52058669607958a98547340ab3032779e977a2af796e3a" gracePeriod=2 Jul 13 21:40:29.643177 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.643151 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb"] Jul 13 21:40:29.646317 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:29.646280 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"55700fca-27bd-4771-b611-7d0787a66ae9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-gvljb\": pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:29.661124 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.661075 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk"] Jul 13 21:40:29.661656 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.661635 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" containerName="manager" Jul 13 21:40:29.661656 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.661656 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" containerName="manager" Jul 13 21:40:29.661791 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.661726 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" containerName="manager" Jul 13 21:40:29.664798 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.664758 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88"] Jul 13 21:40:29.664987 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.664966 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.665075 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.664968 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" containerName="manager" containerID="cri-o://08c15622ecd85dbd64e680be5b2d68da576adf1f1e3e4f761b7c715f08f3d00d" gracePeriod=2 Jul 13 21:40:29.670114 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.670092 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88"] Jul 13 21:40:29.682301 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.682275 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk"] Jul 13 21:40:29.682917 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:29.682885 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:29.686101 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:29.686072 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:29.688385 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:29.688354 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:29.692541 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.692518 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj"] Jul 13 21:40:29.692941 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.692925 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" containerName="manager" Jul 13 21:40:29.693025 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.692943 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" containerName="manager" Jul 13 21:40:29.693079 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.693046 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" containerName="manager" Jul 13 21:40:29.695885 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.695868 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:40:29.715821 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.715799 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj"] Jul 13 21:40:29.726313 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:29.726287 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:29.728842 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:29.728812 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:29.748080 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.748047 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d96p\" (UniqueName: \"kubernetes.io/projected/1365fcd3-4224-4dee-987a-c0913a343bef-kube-api-access-6d96p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppsbk\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.748182 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.748145 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1365fcd3-4224-4dee-987a-c0913a343bef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppsbk\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.849296 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.849247 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84k74\" (UniqueName: \"kubernetes.io/projected/68445109-b151-408b-8ec4-538bb7cae217-kube-api-access-84k74\") pod \"limitador-operator-controller-manager-859d7b78bd-zk7sj\" (UID: \"68445109-b151-408b-8ec4-538bb7cae217\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:40:29.849426 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.849345 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d96p\" (UniqueName: \"kubernetes.io/projected/1365fcd3-4224-4dee-987a-c0913a343bef-kube-api-access-6d96p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppsbk\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.849426 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.849419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1365fcd3-4224-4dee-987a-c0913a343bef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppsbk\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.850053 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.849823 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1365fcd3-4224-4dee-987a-c0913a343bef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppsbk\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.859454 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.859436 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d96p\" (UniqueName: \"kubernetes.io/projected/1365fcd3-4224-4dee-987a-c0913a343bef-kube-api-access-6d96p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppsbk\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:29.879036 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.879005 2583 generic.go:356] "Generic (PLEG): container finished" podID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" containerID="08c15622ecd85dbd64e680be5b2d68da576adf1f1e3e4f761b7c715f08f3d00d" exitCode=0 Jul 13 21:40:29.880951 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.880922 2583 generic.go:356] "Generic (PLEG): container finished" podID="55700fca-27bd-4771-b611-7d0787a66ae9" containerID="91c6114eb23ea37f7f52058669607958a98547340ab3032779e977a2af796e3a" exitCode=0 Jul 13 21:40:29.950078 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.950046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84k74\" (UniqueName: \"kubernetes.io/projected/68445109-b151-408b-8ec4-538bb7cae217-kube-api-access-84k74\") pod \"limitador-operator-controller-manager-859d7b78bd-zk7sj\" (UID: \"68445109-b151-408b-8ec4-538bb7cae217\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:40:29.961752 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:29.961727 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84k74\" (UniqueName: \"kubernetes.io/projected/68445109-b151-408b-8ec4-538bb7cae217-kube-api-access-84k74\") pod \"limitador-operator-controller-manager-859d7b78bd-zk7sj\" (UID: \"68445109-b151-408b-8ec4-538bb7cae217\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:40:30.056625 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:30.056591 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:30.063598 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:30.063573 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:40:30.351944 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:30.351838 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:30.353814 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:30.353765 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:43.570350 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.570292 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:43.572939 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.572906 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:43.574962 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.574934 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:43.577222 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.577136 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:43.579312 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.579282 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:43.677619 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.677587 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw2gj\" (UniqueName: \"kubernetes.io/projected/d27ff499-3ca0-423b-b5a4-9e078c1ef589-kube-api-access-bw2gj\") pod \"d27ff499-3ca0-423b-b5a4-9e078c1ef589\" (UID: \"d27ff499-3ca0-423b-b5a4-9e078c1ef589\") " Jul 13 21:40:43.677740 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.677667 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55700fca-27bd-4771-b611-7d0787a66ae9-extensions-socket-volume\") pod \"55700fca-27bd-4771-b611-7d0787a66ae9\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " Jul 13 21:40:43.677829 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.677756 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbrlf\" (UniqueName: \"kubernetes.io/projected/55700fca-27bd-4771-b611-7d0787a66ae9-kube-api-access-gbrlf\") pod \"55700fca-27bd-4771-b611-7d0787a66ae9\" (UID: \"55700fca-27bd-4771-b611-7d0787a66ae9\") " Jul 13 21:40:43.678423 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.678281 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55700fca-27bd-4771-b611-7d0787a66ae9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "55700fca-27bd-4771-b611-7d0787a66ae9" (UID: "55700fca-27bd-4771-b611-7d0787a66ae9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:40:43.679708 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.679683 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d27ff499-3ca0-423b-b5a4-9e078c1ef589-kube-api-access-bw2gj" (OuterVolumeSpecName: "kube-api-access-bw2gj") pod "d27ff499-3ca0-423b-b5a4-9e078c1ef589" (UID: "d27ff499-3ca0-423b-b5a4-9e078c1ef589"). InnerVolumeSpecName "kube-api-access-bw2gj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:40:43.680174 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.680145 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55700fca-27bd-4771-b611-7d0787a66ae9-kube-api-access-gbrlf" (OuterVolumeSpecName: "kube-api-access-gbrlf") pod "55700fca-27bd-4771-b611-7d0787a66ae9" (UID: "55700fca-27bd-4771-b611-7d0787a66ae9"). InnerVolumeSpecName "kube-api-access-gbrlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:40:43.687901 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.687879 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk"] Jul 13 21:40:43.689909 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:40:43.689886 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1365fcd3_4224_4dee_987a_c0913a343bef.slice/crio-c26617ae54d6e6450b66a5fcf16b86cad9786716e01f0c68951797208acf3e8a WatchSource:0}: Error finding container c26617ae54d6e6450b66a5fcf16b86cad9786716e01f0c68951797208acf3e8a: Status 404 returned error can't find the container with id c26617ae54d6e6450b66a5fcf16b86cad9786716e01f0c68951797208acf3e8a Jul 13 21:40:43.705065 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.705041 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj"] Jul 13 21:40:43.705589 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:40:43.705566 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68445109_b151_408b_8ec4_538bb7cae217.slice/crio-e8c9403b0aacbbb64a533b27d6f3ac19a9a92a2ea59f2bb0cbd1ceb916bfd675 WatchSource:0}: Error finding container e8c9403b0aacbbb64a533b27d6f3ac19a9a92a2ea59f2bb0cbd1ceb916bfd675: Status 404 returned error can't find the container with id e8c9403b0aacbbb64a533b27d6f3ac19a9a92a2ea59f2bb0cbd1ceb916bfd675 Jul 13 21:40:43.779147 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.779124 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw2gj\" (UniqueName: \"kubernetes.io/projected/d27ff499-3ca0-423b-b5a4-9e078c1ef589-kube-api-access-bw2gj\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:40:43.779147 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.779147 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55700fca-27bd-4771-b611-7d0787a66ae9-extensions-socket-volume\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:40:43.779282 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.779157 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbrlf\" (UniqueName: \"kubernetes.io/projected/55700fca-27bd-4771-b611-7d0787a66ae9-kube-api-access-gbrlf\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:40:43.963142 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.963119 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:43.963284 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.963130 2583 scope.go:117] "RemoveContainer" containerID="08c15622ecd85dbd64e680be5b2d68da576adf1f1e3e4f761b7c715f08f3d00d" Jul 13 21:40:43.964838 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.964808 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" event={"ID":"02ef0294-96c7-45be-94d4-76d120d6e6b1","Type":"ContainerStarted","Data":"3f92eab643eabf4317b7b3dd981e62708e2d4f4f8da59b90d2ce8244ac328810"} Jul 13 21:40:43.965436 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.965402 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:43.966819 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.966792 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" event={"ID":"68445109-b151-408b-8ec4-538bb7cae217","Type":"ContainerStarted","Data":"c0c2e250f42588d9401920982a9059e3d1994f972c137806d90cb450a34f8a72"} Jul 13 21:40:43.966943 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.966845 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" event={"ID":"68445109-b151-408b-8ec4-538bb7cae217","Type":"ContainerStarted","Data":"e8c9403b0aacbbb64a533b27d6f3ac19a9a92a2ea59f2bb0cbd1ceb916bfd675"} Jul 13 21:40:43.966943 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.966872 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:40:43.967526 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.967494 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:43.968430 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.968410 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:43.970072 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.970048 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" event={"ID":"1365fcd3-4224-4dee-987a-c0913a343bef","Type":"ContainerStarted","Data":"4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a"} Jul 13 21:40:43.970180 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.970089 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" event={"ID":"1365fcd3-4224-4dee-987a-c0913a343bef","Type":"ContainerStarted","Data":"c26617ae54d6e6450b66a5fcf16b86cad9786716e01f0c68951797208acf3e8a"} Jul 13 21:40:43.970246 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.970193 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:43.974092 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.973726 2583 scope.go:117] "RemoveContainer" containerID="91c6114eb23ea37f7f52058669607958a98547340ab3032779e977a2af796e3a" Jul 13 21:40:43.986110 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:43.986071 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" podStartSLOduration=14.986057834 podStartE2EDuration="14.986057834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:43.984082192 +0000 UTC m=+664.803592028" watchObservedRunningTime="2026-07-13 21:40:43.986057834 +0000 UTC m=+664.805567643" Jul 13 21:40:43.986437 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.986410 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:43.988444 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:43.988421 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:44.005282 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:44.005237 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4s9mw" podStartSLOduration=1.744213213 podStartE2EDuration="25.00522554s" totalImagesPullingTime="23.261012327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:44.00504518 +0000 UTC m=+664.824554992" watchObservedRunningTime="2026-07-13 21:40:44.00522554 +0000 UTC m=+664.824735349" Jul 13 21:40:44.007191 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:44.007165 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b7p88\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b7p88" Jul 13 21:40:44.009160 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:44.009138 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gvljb\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gvljb" Jul 13 21:40:44.030404 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:44.030356 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" podStartSLOduration=15.030344159 podStartE2EDuration="15.030344159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:44.028011646 +0000 UTC m=+664.847521455" watchObservedRunningTime="2026-07-13 21:40:44.030344159 +0000 UTC m=+664.849853968" Jul 13 21:40:44.351315 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:44.351244 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55700fca-27bd-4771-b611-7d0787a66ae9" path="/var/lib/kubelet/pods/55700fca-27bd-4771-b611-7d0787a66ae9/volumes" Jul 13 21:40:44.351560 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:44.351548 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d27ff499-3ca0-423b-b5a4-9e078c1ef589" path="/var/lib/kubelet/pods/d27ff499-3ca0-423b-b5a4-9e078c1ef589/volumes" Jul 13 21:40:45.840397 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:45.840363 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk"] Jul 13 21:40:45.982104 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:45.982061 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" podUID="1365fcd3-4224-4dee-987a-c0913a343bef" containerName="manager" containerID="cri-o://4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a" gracePeriod=10 Jul 13 21:40:46.226411 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.226389 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:46.399953 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.399882 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d96p\" (UniqueName: \"kubernetes.io/projected/1365fcd3-4224-4dee-987a-c0913a343bef-kube-api-access-6d96p\") pod \"1365fcd3-4224-4dee-987a-c0913a343bef\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " Jul 13 21:40:46.400110 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.399953 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1365fcd3-4224-4dee-987a-c0913a343bef-extensions-socket-volume\") pod \"1365fcd3-4224-4dee-987a-c0913a343bef\" (UID: \"1365fcd3-4224-4dee-987a-c0913a343bef\") " Jul 13 21:40:46.400362 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.400337 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1365fcd3-4224-4dee-987a-c0913a343bef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1365fcd3-4224-4dee-987a-c0913a343bef" (UID: "1365fcd3-4224-4dee-987a-c0913a343bef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:40:46.401949 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.401925 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1365fcd3-4224-4dee-987a-c0913a343bef-kube-api-access-6d96p" (OuterVolumeSpecName: "kube-api-access-6d96p") pod "1365fcd3-4224-4dee-987a-c0913a343bef" (UID: "1365fcd3-4224-4dee-987a-c0913a343bef"). InnerVolumeSpecName "kube-api-access-6d96p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:40:46.500663 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.500633 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d96p\" (UniqueName: \"kubernetes.io/projected/1365fcd3-4224-4dee-987a-c0913a343bef-kube-api-access-6d96p\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:40:46.500663 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.500659 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1365fcd3-4224-4dee-987a-c0913a343bef-extensions-socket-volume\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:40:46.988898 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.988856 2583 generic.go:356] "Generic (PLEG): container finished" podID="1365fcd3-4224-4dee-987a-c0913a343bef" containerID="4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a" exitCode=0 Jul 13 21:40:46.989378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.988909 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" event={"ID":"1365fcd3-4224-4dee-987a-c0913a343bef","Type":"ContainerDied","Data":"4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a"} Jul 13 21:40:46.989378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.988954 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" event={"ID":"1365fcd3-4224-4dee-987a-c0913a343bef","Type":"ContainerDied","Data":"c26617ae54d6e6450b66a5fcf16b86cad9786716e01f0c68951797208acf3e8a"} Jul 13 21:40:46.989378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.988976 2583 scope.go:117] "RemoveContainer" containerID="4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a" Jul 13 21:40:46.989378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:46.988910 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk" Jul 13 21:40:47.002061 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:47.002028 2583 scope.go:117] "RemoveContainer" containerID="4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a" Jul 13 21:40:47.002331 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:40:47.002310 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a\": container with ID starting with 4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a not found: ID does not exist" containerID="4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a" Jul 13 21:40:47.002377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:47.002357 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a"} err="failed to get container status \"4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a\": rpc error: code = NotFound desc = could not find container \"4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a\": container with ID starting with 4e0a59ca59fe08e9279375b826b052322560c6759f5b11693fd0aeb7e298368a not found: ID does not exist" Jul 13 21:40:47.014792 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:47.014745 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk"] Jul 13 21:40:47.019059 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:47.019036 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppsbk"] Jul 13 21:40:48.351603 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:48.351571 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1365fcd3-4224-4dee-987a-c0913a343bef" path="/var/lib/kubelet/pods/1365fcd3-4224-4dee-987a-c0913a343bef/volumes" Jul 13 21:40:54.980359 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:40:54.980323 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zk7sj" Jul 13 21:41:02.259434 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.259397 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7"] Jul 13 21:41:02.259957 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.259825 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1365fcd3-4224-4dee-987a-c0913a343bef" containerName="manager" Jul 13 21:41:02.259957 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.259843 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1365fcd3-4224-4dee-987a-c0913a343bef" containerName="manager" Jul 13 21:41:02.259957 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.259940 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1365fcd3-4224-4dee-987a-c0913a343bef" containerName="manager" Jul 13 21:41:02.349695 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.349663 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.352272 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.352245 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g25bp\"" Jul 13 21:41:02.356652 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.356629 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7"] Jul 13 21:41:02.415259 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415372 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415270 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415372 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415329 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415456 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415456 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415456 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415565 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415565 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415488 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2npfc\" (UniqueName: \"kubernetes.io/projected/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-kube-api-access-2npfc\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.415565 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.415535 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516212 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516212 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2npfc\" (UniqueName: \"kubernetes.io/projected/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-kube-api-access-2npfc\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516284 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516315 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516377 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516659 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516906 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516716 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.516906 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.516852 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.517184 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.517162 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.518608 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.518585 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.518817 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.518802 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.524597 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.524570 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.524815 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.524796 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2npfc\" (UniqueName: \"kubernetes.io/projected/cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b-kube-api-access-2npfc\") pod \"maas-default-gateway-openshift-default-56654dcf7-tfxd7\" (UID: \"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.661824 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.661791 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:02.811418 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.811354 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7"] Jul 13 21:41:02.814017 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:02.813988 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3b4a1a_acbb_4915_9e03_94f5ccfa3a4b.slice/crio-1ca7b2a053309218d8b8541198232067073570ec356445e954cc98c3d1354994 WatchSource:0}: Error finding container 1ca7b2a053309218d8b8541198232067073570ec356445e954cc98c3d1354994: Status 404 returned error can't find the container with id 1ca7b2a053309218d8b8541198232067073570ec356445e954cc98c3d1354994 Jul 13 21:41:02.816165 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.816135 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:41:02.816257 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.816197 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:41:02.816257 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:02.816224 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:41:03.075559 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:03.075480 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" event={"ID":"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b","Type":"ContainerStarted","Data":"b723e8760b4d4bdb3ee15937ca69d4541b3d6983ed4252743c9c57705be0a1ff"} Jul 13 21:41:03.075559 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:03.075531 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" event={"ID":"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b","Type":"ContainerStarted","Data":"1ca7b2a053309218d8b8541198232067073570ec356445e954cc98c3d1354994"} Jul 13 21:41:03.094252 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:03.094197 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" podStartSLOduration=1.09418043 podStartE2EDuration="1.09418043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:03.093100382 +0000 UTC m=+683.912610194" watchObservedRunningTime="2026-07-13 21:41:03.09418043 +0000 UTC m=+683.913690238" Jul 13 21:41:03.662166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:03.662130 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:03.666823 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:03.666798 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:04.080892 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:04.080849 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:04.082069 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:04.082044 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:41:18.849378 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.849347 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:18.879619 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.879592 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:18.879752 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.879683 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:18.882216 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.882193 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 21:41:18.943924 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.943876 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmdpc\" (UniqueName: \"kubernetes.io/projected/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-kube-api-access-wmdpc\") pod \"limitador-limitador-64c94fd64f-whbn5\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:18.944078 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.943945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-config-file\") pod \"limitador-limitador-64c94fd64f-whbn5\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:18.950014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:18.949990 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:19.044481 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.044453 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-config-file\") pod \"limitador-limitador-64c94fd64f-whbn5\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:19.044623 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.044567 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdpc\" (UniqueName: \"kubernetes.io/projected/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-kube-api-access-wmdpc\") pod \"limitador-limitador-64c94fd64f-whbn5\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:19.045192 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.045170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-config-file\") pod \"limitador-limitador-64c94fd64f-whbn5\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:19.055164 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.055131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdpc\" (UniqueName: \"kubernetes.io/projected/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-kube-api-access-wmdpc\") pod \"limitador-limitador-64c94fd64f-whbn5\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:19.189832 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.189806 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:19.322241 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.322219 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:19.323666 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:19.323633 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe04f0b2_d2be_464a_a5dc_6e5350e53a7b.slice/crio-5677096edd3046b5930ec11fdde7f13272a5a30d46e293b9f47909788f70f545 WatchSource:0}: Error finding container 5677096edd3046b5930ec11fdde7f13272a5a30d46e293b9f47909788f70f545: Status 404 returned error can't find the container with id 5677096edd3046b5930ec11fdde7f13272a5a30d46e293b9f47909788f70f545 Jul 13 21:41:19.325835 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.325810 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:41:19.806877 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.806850 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-lkg8m"] Jul 13 21:41:19.811473 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.811457 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:19.813928 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.813909 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cpbz4\"" Jul 13 21:41:19.817159 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.817138 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lkg8m"] Jul 13 21:41:19.851299 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.851276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mss7j\" (UniqueName: \"kubernetes.io/projected/984e2a17-91d0-49a3-af2a-9864912974fa-kube-api-access-mss7j\") pod \"authorino-75d556bd77-lkg8m\" (UID: \"984e2a17-91d0-49a3-af2a-9864912974fa\") " pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:19.952601 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.952575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mss7j\" (UniqueName: \"kubernetes.io/projected/984e2a17-91d0-49a3-af2a-9864912974fa-kube-api-access-mss7j\") pod \"authorino-75d556bd77-lkg8m\" (UID: \"984e2a17-91d0-49a3-af2a-9864912974fa\") " pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:19.960610 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:19.960590 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mss7j\" (UniqueName: \"kubernetes.io/projected/984e2a17-91d0-49a3-af2a-9864912974fa-kube-api-access-mss7j\") pod \"authorino-75d556bd77-lkg8m\" (UID: \"984e2a17-91d0-49a3-af2a-9864912974fa\") " pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:20.121836 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:20.121607 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:20.179042 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:20.179007 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" event={"ID":"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b","Type":"ContainerStarted","Data":"5677096edd3046b5930ec11fdde7f13272a5a30d46e293b9f47909788f70f545"} Jul 13 21:41:20.292653 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:20.292620 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lkg8m"] Jul 13 21:41:20.295260 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:20.295227 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod984e2a17_91d0_49a3_af2a_9864912974fa.slice/crio-b4fd4c4c732db5325e9c0789c09e154fbef5b12e5d4dbd2eacb6eb5ac780dd2c WatchSource:0}: Error finding container b4fd4c4c732db5325e9c0789c09e154fbef5b12e5d4dbd2eacb6eb5ac780dd2c: Status 404 returned error can't find the container with id b4fd4c4c732db5325e9c0789c09e154fbef5b12e5d4dbd2eacb6eb5ac780dd2c Jul 13 21:41:21.187068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:21.187027 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lkg8m" event={"ID":"984e2a17-91d0-49a3-af2a-9864912974fa","Type":"ContainerStarted","Data":"b4fd4c4c732db5325e9c0789c09e154fbef5b12e5d4dbd2eacb6eb5ac780dd2c"} Jul 13 21:41:22.196979 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:22.196943 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" event={"ID":"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b","Type":"ContainerStarted","Data":"0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43"} Jul 13 21:41:22.197319 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:22.197048 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:22.212812 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:22.212227 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" podStartSLOduration=1.411998136 podStartE2EDuration="4.212206769s" totalImagesPullingTime="2.800208633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:22.211037997 +0000 UTC m=+703.030547827" watchObservedRunningTime="2026-07-13 21:41:22.212206769 +0000 UTC m=+703.031716579" Jul 13 21:41:25.216463 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:25.216426 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lkg8m" event={"ID":"984e2a17-91d0-49a3-af2a-9864912974fa","Type":"ContainerStarted","Data":"ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0"} Jul 13 21:41:25.232975 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:25.232928 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-lkg8m" podStartSLOduration=2.133682482 podStartE2EDuration="6.232913688s" totalImagesPullingTime="4.099231206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:25.231612738 +0000 UTC m=+706.051122547" watchObservedRunningTime="2026-07-13 21:41:25.232913688 +0000 UTC m=+706.052423503" Jul 13 21:41:33.203923 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:33.203841 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:34.046756 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.046718 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:34.047121 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.047076 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" podUID="fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" containerName="limitador" containerID="cri-o://0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43" gracePeriod=30 Jul 13 21:41:34.594543 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.594517 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:34.670575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.670512 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmdpc\" (UniqueName: \"kubernetes.io/projected/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-kube-api-access-wmdpc\") pod \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " Jul 13 21:41:34.670575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.670542 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-config-file\") pod \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\" (UID: \"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b\") " Jul 13 21:41:34.670937 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.670912 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-config-file" (OuterVolumeSpecName: "config-file") pod "fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" (UID: "fe04f0b2-d2be-464a-a5dc-6e5350e53a7b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:41:34.672469 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.672446 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-kube-api-access-wmdpc" (OuterVolumeSpecName: "kube-api-access-wmdpc") pod "fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" (UID: "fe04f0b2-d2be-464a-a5dc-6e5350e53a7b"). InnerVolumeSpecName "kube-api-access-wmdpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:41:34.771529 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.771506 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmdpc\" (UniqueName: \"kubernetes.io/projected/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-kube-api-access-wmdpc\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:41:34.771529 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:34.771527 2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b-config-file\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:41:35.277899 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.277866 2583 generic.go:356] "Generic (PLEG): container finished" podID="fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" containerID="0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43" exitCode=0 Jul 13 21:41:35.278091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.277920 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" Jul 13 21:41:35.278091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.277954 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" event={"ID":"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b","Type":"ContainerDied","Data":"0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43"} Jul 13 21:41:35.278091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.278006 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-whbn5" event={"ID":"fe04f0b2-d2be-464a-a5dc-6e5350e53a7b","Type":"ContainerDied","Data":"5677096edd3046b5930ec11fdde7f13272a5a30d46e293b9f47909788f70f545"} Jul 13 21:41:35.278091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.278029 2583 scope.go:117] "RemoveContainer" containerID="0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43" Jul 13 21:41:35.286475 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.286457 2583 scope.go:117] "RemoveContainer" containerID="0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43" Jul 13 21:41:35.286719 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:41:35.286702 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43\": container with ID starting with 0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43 not found: ID does not exist" containerID="0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43" Jul 13 21:41:35.286757 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.286728 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43"} err="failed to get container status \"0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43\": rpc error: code = NotFound desc = could not find container \"0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43\": container with ID starting with 0255586638a7194116b1151df187fe233dce58708ad28584f4297b23ca7c9e43 not found: ID does not exist" Jul 13 21:41:35.299029 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.299004 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:35.300979 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:35.300963 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-whbn5"] Jul 13 21:41:36.310630 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.310592 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk"] Jul 13 21:41:36.311109 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.311089 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" containerName="limitador" Jul 13 21:41:36.311181 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.311112 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" containerName="limitador" Jul 13 21:41:36.311246 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.311234 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" containerName="limitador" Jul 13 21:41:36.315236 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.315216 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.317870 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.317849 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v8wq9\"" Jul 13 21:41:36.317981 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.317905 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 21:41:36.317981 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.317975 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 21:41:36.319014 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.318952 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 21:41:36.321593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.321571 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk"] Jul 13 21:41:36.351333 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.351310 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe04f0b2-d2be-464a-a5dc-6e5350e53a7b" path="/var/lib/kubelet/pods/fe04f0b2-d2be-464a-a5dc-6e5350e53a7b/volumes" Jul 13 21:41:36.383811 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.383789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/18b55dab-ad2f-4ab7-a464-b1f7be2052ae-data\") pod \"postgres-69fb4594cf-5r5hk\" (UID: \"18b55dab-ad2f-4ab7-a464-b1f7be2052ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.383900 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.383824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54bqd\" (UniqueName: \"kubernetes.io/projected/18b55dab-ad2f-4ab7-a464-b1f7be2052ae-kube-api-access-54bqd\") pod \"postgres-69fb4594cf-5r5hk\" (UID: \"18b55dab-ad2f-4ab7-a464-b1f7be2052ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.484601 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.484572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/18b55dab-ad2f-4ab7-a464-b1f7be2052ae-data\") pod \"postgres-69fb4594cf-5r5hk\" (UID: \"18b55dab-ad2f-4ab7-a464-b1f7be2052ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.484725 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.484637 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54bqd\" (UniqueName: \"kubernetes.io/projected/18b55dab-ad2f-4ab7-a464-b1f7be2052ae-kube-api-access-54bqd\") pod \"postgres-69fb4594cf-5r5hk\" (UID: \"18b55dab-ad2f-4ab7-a464-b1f7be2052ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.484982 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.484962 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/18b55dab-ad2f-4ab7-a464-b1f7be2052ae-data\") pod \"postgres-69fb4594cf-5r5hk\" (UID: \"18b55dab-ad2f-4ab7-a464-b1f7be2052ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.492443 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.492423 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54bqd\" (UniqueName: \"kubernetes.io/projected/18b55dab-ad2f-4ab7-a464-b1f7be2052ae-kube-api-access-54bqd\") pod \"postgres-69fb4594cf-5r5hk\" (UID: \"18b55dab-ad2f-4ab7-a464-b1f7be2052ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.626888 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.626834 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:36.755450 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:36.755420 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk"] Jul 13 21:41:36.756843 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:36.756818 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b55dab_ad2f_4ab7_a464_b1f7be2052ae.slice/crio-3d58f368a0a47a86bce9c066aaeddeb29d53872da9d3a8630482b26fda8f3fe2 WatchSource:0}: Error finding container 3d58f368a0a47a86bce9c066aaeddeb29d53872da9d3a8630482b26fda8f3fe2: Status 404 returned error can't find the container with id 3d58f368a0a47a86bce9c066aaeddeb29d53872da9d3a8630482b26fda8f3fe2 Jul 13 21:41:37.290236 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:37.290201 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" event={"ID":"18b55dab-ad2f-4ab7-a464-b1f7be2052ae","Type":"ContainerStarted","Data":"3d58f368a0a47a86bce9c066aaeddeb29d53872da9d3a8630482b26fda8f3fe2"} Jul 13 21:41:41.479372 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:41.479350 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 21:41:42.326707 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:42.326677 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" event={"ID":"18b55dab-ad2f-4ab7-a464-b1f7be2052ae","Type":"ContainerStarted","Data":"0eebae5882d9415b9b35a6c6fc262e0a16e01ce7c88229524c217cf2fd16937f"} Jul 13 21:41:42.326895 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:42.326745 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:42.344068 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:42.344024 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" podStartSLOduration=1.625812295 podStartE2EDuration="6.3440098s" totalImagesPullingTime="4.718197505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:42.341435379 +0000 UTC m=+723.160945208" watchObservedRunningTime="2026-07-13 21:41:42.3440098 +0000 UTC m=+723.163519608" Jul 13 21:41:48.360623 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:48.360585 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5r5hk" Jul 13 21:41:48.936134 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:48.936099 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc64w"] Jul 13 21:41:48.942491 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:48.942469 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:48.945339 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:48.945316 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc64w"] Jul 13 21:41:49.081393 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.081369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55nls\" (UniqueName: \"kubernetes.io/projected/2984f302-3256-4c20-ae92-fb4b14f7c732-kube-api-access-55nls\") pod \"authorino-5d4b67c446-fc64w\" (UID: \"2984f302-3256-4c20-ae92-fb4b14f7c732\") " pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:49.182329 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.182301 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55nls\" (UniqueName: \"kubernetes.io/projected/2984f302-3256-4c20-ae92-fb4b14f7c732-kube-api-access-55nls\") pod \"authorino-5d4b67c446-fc64w\" (UID: \"2984f302-3256-4c20-ae92-fb4b14f7c732\") " pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:49.190555 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.190499 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55nls\" (UniqueName: \"kubernetes.io/projected/2984f302-3256-4c20-ae92-fb4b14f7c732-kube-api-access-55nls\") pod \"authorino-5d4b67c446-fc64w\" (UID: \"2984f302-3256-4c20-ae92-fb4b14f7c732\") " pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:49.191835 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.191815 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc64w"] Jul 13 21:41:49.191995 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.191984 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:49.220838 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.220806 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64fb7789d7-jc24c"] Jul 13 21:41:49.226194 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.226175 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:49.230862 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.230840 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64fb7789d7-jc24c"] Jul 13 21:41:49.327761 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.327739 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc64w"] Jul 13 21:41:49.329607 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:49.329583 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2984f302_3256_4c20_ae92_fb4b14f7c732.slice/crio-5ee7b80e13f1a020fd4ec5cceec4f44b73702a88bad42b2f5fed0a0ec05cc326 WatchSource:0}: Error finding container 5ee7b80e13f1a020fd4ec5cceec4f44b73702a88bad42b2f5fed0a0ec05cc326: Status 404 returned error can't find the container with id 5ee7b80e13f1a020fd4ec5cceec4f44b73702a88bad42b2f5fed0a0ec05cc326 Jul 13 21:41:49.361949 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.361924 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fc64w" event={"ID":"2984f302-3256-4c20-ae92-fb4b14f7c732","Type":"ContainerStarted","Data":"5ee7b80e13f1a020fd4ec5cceec4f44b73702a88bad42b2f5fed0a0ec05cc326"} Jul 13 21:41:49.384314 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.384293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjchd\" (UniqueName: \"kubernetes.io/projected/0aeb449d-2145-48b7-b71c-75dfb6a04a96-kube-api-access-sjchd\") pod \"authorino-64fb7789d7-jc24c\" (UID: \"0aeb449d-2145-48b7-b71c-75dfb6a04a96\") " pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:49.484987 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.484926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjchd\" (UniqueName: \"kubernetes.io/projected/0aeb449d-2145-48b7-b71c-75dfb6a04a96-kube-api-access-sjchd\") pod \"authorino-64fb7789d7-jc24c\" (UID: \"0aeb449d-2145-48b7-b71c-75dfb6a04a96\") " pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:49.492879 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.492851 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjchd\" (UniqueName: \"kubernetes.io/projected/0aeb449d-2145-48b7-b71c-75dfb6a04a96-kube-api-access-sjchd\") pod \"authorino-64fb7789d7-jc24c\" (UID: \"0aeb449d-2145-48b7-b71c-75dfb6a04a96\") " pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:49.536521 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.536501 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:49.594099 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.593135 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64fb7789d7-jc24c"] Jul 13 21:41:49.620206 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.620160 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f7dcf78f9-594pr"] Jul 13 21:41:49.625849 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.625825 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.628154 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.628134 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 13 21:41:49.636691 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.636666 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f7dcf78f9-594pr"] Jul 13 21:41:49.671478 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.671458 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64fb7789d7-jc24c"] Jul 13 21:41:49.674558 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:49.674535 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aeb449d_2145_48b7_b71c_75dfb6a04a96.slice/crio-25480af6a836301066513a910b1d5f65c15a6068ea76b50966a5ef65e787c8d4 WatchSource:0}: Error finding container 25480af6a836301066513a910b1d5f65c15a6068ea76b50966a5ef65e787c8d4: Status 404 returned error can't find the container with id 25480af6a836301066513a910b1d5f65c15a6068ea76b50966a5ef65e787c8d4 Jul 13 21:41:49.789214 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.789135 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97be81df-c253-4af5-940b-0e413a704b54-tls-cert\") pod \"authorino-7f7dcf78f9-594pr\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.789214 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.789171 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b98vc\" (UniqueName: \"kubernetes.io/projected/97be81df-c253-4af5-940b-0e413a704b54-kube-api-access-b98vc\") pod \"authorino-7f7dcf78f9-594pr\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.890336 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.890309 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97be81df-c253-4af5-940b-0e413a704b54-tls-cert\") pod \"authorino-7f7dcf78f9-594pr\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.890465 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.890342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b98vc\" (UniqueName: \"kubernetes.io/projected/97be81df-c253-4af5-940b-0e413a704b54-kube-api-access-b98vc\") pod \"authorino-7f7dcf78f9-594pr\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.892701 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.892682 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97be81df-c253-4af5-940b-0e413a704b54-tls-cert\") pod \"authorino-7f7dcf78f9-594pr\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.898216 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.898195 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b98vc\" (UniqueName: \"kubernetes.io/projected/97be81df-c253-4af5-940b-0e413a704b54-kube-api-access-b98vc\") pod \"authorino-7f7dcf78f9-594pr\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:49.941291 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:49.941271 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:41:50.066331 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.066306 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f7dcf78f9-594pr"] Jul 13 21:41:50.067993 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:50.067970 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97be81df_c253_4af5_940b_0e413a704b54.slice/crio-2ca99051630a7dcd0251e19ef3cbd6365ca5c99c7c4ff29209ee1f8019ba2f4f WatchSource:0}: Error finding container 2ca99051630a7dcd0251e19ef3cbd6365ca5c99c7c4ff29209ee1f8019ba2f4f: Status 404 returned error can't find the container with id 2ca99051630a7dcd0251e19ef3cbd6365ca5c99c7c4ff29209ee1f8019ba2f4f Jul 13 21:41:50.368247 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.368148 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fc64w" event={"ID":"2984f302-3256-4c20-ae92-fb4b14f7c732","Type":"ContainerStarted","Data":"99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441"} Jul 13 21:41:50.368247 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.368186 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fc64w" podUID="2984f302-3256-4c20-ae92-fb4b14f7c732" containerName="authorino" containerID="cri-o://99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441" gracePeriod=30 Jul 13 21:41:50.369412 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.369389 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" event={"ID":"97be81df-c253-4af5-940b-0e413a704b54","Type":"ContainerStarted","Data":"2ca99051630a7dcd0251e19ef3cbd6365ca5c99c7c4ff29209ee1f8019ba2f4f"} Jul 13 21:41:50.370955 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.370933 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fb7789d7-jc24c" event={"ID":"0aeb449d-2145-48b7-b71c-75dfb6a04a96","Type":"ContainerStarted","Data":"caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c"} Jul 13 21:41:50.371090 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.370954 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-64fb7789d7-jc24c" podUID="0aeb449d-2145-48b7-b71c-75dfb6a04a96" containerName="authorino" containerID="cri-o://caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c" gracePeriod=30 Jul 13 21:41:50.371090 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.370963 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fb7789d7-jc24c" event={"ID":"0aeb449d-2145-48b7-b71c-75dfb6a04a96","Type":"ContainerStarted","Data":"25480af6a836301066513a910b1d5f65c15a6068ea76b50966a5ef65e787c8d4"} Jul 13 21:41:50.386732 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.386693 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fc64w" podStartSLOduration=2.05499819 podStartE2EDuration="2.386679079s" totalImagesPullingTime="331.680889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:50.383851531 +0000 UTC m=+731.203361340" watchObservedRunningTime="2026-07-13 21:41:50.386679079 +0000 UTC m=+731.206188940" Jul 13 21:41:50.398510 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.398472 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64fb7789d7-jc24c" podStartSLOduration=0.983813897 podStartE2EDuration="1.398459433s" totalImagesPullingTime="414.645536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:50.396901279 +0000 UTC m=+731.216411090" watchObservedRunningTime="2026-07-13 21:41:50.398459433 +0000 UTC m=+731.217969241" Jul 13 21:41:50.647681 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.647657 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:50.651693 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.651670 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:50.798519 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.798485 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55nls\" (UniqueName: \"kubernetes.io/projected/2984f302-3256-4c20-ae92-fb4b14f7c732-kube-api-access-55nls\") pod \"2984f302-3256-4c20-ae92-fb4b14f7c732\" (UID: \"2984f302-3256-4c20-ae92-fb4b14f7c732\") " Jul 13 21:41:50.798669 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.798590 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjchd\" (UniqueName: \"kubernetes.io/projected/0aeb449d-2145-48b7-b71c-75dfb6a04a96-kube-api-access-sjchd\") pod \"0aeb449d-2145-48b7-b71c-75dfb6a04a96\" (UID: \"0aeb449d-2145-48b7-b71c-75dfb6a04a96\") " Jul 13 21:41:50.800479 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.800450 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2984f302-3256-4c20-ae92-fb4b14f7c732-kube-api-access-55nls" (OuterVolumeSpecName: "kube-api-access-55nls") pod "2984f302-3256-4c20-ae92-fb4b14f7c732" (UID: "2984f302-3256-4c20-ae92-fb4b14f7c732"). InnerVolumeSpecName "kube-api-access-55nls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:41:50.800588 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.800450 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aeb449d-2145-48b7-b71c-75dfb6a04a96-kube-api-access-sjchd" (OuterVolumeSpecName: "kube-api-access-sjchd") pod "0aeb449d-2145-48b7-b71c-75dfb6a04a96" (UID: "0aeb449d-2145-48b7-b71c-75dfb6a04a96"). InnerVolumeSpecName "kube-api-access-sjchd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:41:50.899558 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.899500 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjchd\" (UniqueName: \"kubernetes.io/projected/0aeb449d-2145-48b7-b71c-75dfb6a04a96-kube-api-access-sjchd\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:41:50.899558 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:50.899521 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55nls\" (UniqueName: \"kubernetes.io/projected/2984f302-3256-4c20-ae92-fb4b14f7c732-kube-api-access-55nls\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:41:51.377864 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.377829 2583 generic.go:356] "Generic (PLEG): container finished" podID="2984f302-3256-4c20-ae92-fb4b14f7c732" containerID="99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441" exitCode=0 Jul 13 21:41:51.377864 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.377864 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc64w" Jul 13 21:41:51.378302 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.377914 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fc64w" event={"ID":"2984f302-3256-4c20-ae92-fb4b14f7c732","Type":"ContainerDied","Data":"99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441"} Jul 13 21:41:51.378302 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.377948 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fc64w" event={"ID":"2984f302-3256-4c20-ae92-fb4b14f7c732","Type":"ContainerDied","Data":"5ee7b80e13f1a020fd4ec5cceec4f44b73702a88bad42b2f5fed0a0ec05cc326"} Jul 13 21:41:51.378302 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.377968 2583 scope.go:117] "RemoveContainer" containerID="99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441" Jul 13 21:41:51.379609 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.379583 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" event={"ID":"97be81df-c253-4af5-940b-0e413a704b54","Type":"ContainerStarted","Data":"05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d"} Jul 13 21:41:51.380860 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.380836 2583 generic.go:356] "Generic (PLEG): container finished" podID="0aeb449d-2145-48b7-b71c-75dfb6a04a96" containerID="caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c" exitCode=0 Jul 13 21:41:51.380954 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.380888 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fb7789d7-jc24c" Jul 13 21:41:51.381020 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.380890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fb7789d7-jc24c" event={"ID":"0aeb449d-2145-48b7-b71c-75dfb6a04a96","Type":"ContainerDied","Data":"caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c"} Jul 13 21:41:51.381075 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.381022 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fb7789d7-jc24c" event={"ID":"0aeb449d-2145-48b7-b71c-75dfb6a04a96","Type":"ContainerDied","Data":"25480af6a836301066513a910b1d5f65c15a6068ea76b50966a5ef65e787c8d4"} Jul 13 21:41:51.387524 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.387506 2583 scope.go:117] "RemoveContainer" containerID="99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441" Jul 13 21:41:51.387760 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:41:51.387741 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441\": container with ID starting with 99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441 not found: ID does not exist" containerID="99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441" Jul 13 21:41:51.387844 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.387795 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441"} err="failed to get container status \"99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441\": rpc error: code = NotFound desc = could not find container \"99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441\": container with ID starting with 99330e04766ec55fe93914434d7611506beb3c80973de5981e02560e1548e441 not found: ID does not exist" Jul 13 21:41:51.387844 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.387822 2583 scope.go:117] "RemoveContainer" containerID="caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c" Jul 13 21:41:51.395488 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.395473 2583 scope.go:117] "RemoveContainer" containerID="caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c" Jul 13 21:41:51.395860 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:41:51.395831 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c\": container with ID starting with caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c not found: ID does not exist" containerID="caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c" Jul 13 21:41:51.395951 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.395863 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c"} err="failed to get container status \"caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c\": rpc error: code = NotFound desc = could not find container \"caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c\": container with ID starting with caf69746e8ade04c89196b4fb1c0c463a0f7d097f7029b70ca07b4c971a27f2c not found: ID does not exist" Jul 13 21:41:51.398336 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.398297 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" podStartSLOduration=2.0156872200000002 podStartE2EDuration="2.398284609s" totalImagesPullingTime="382.597389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:51.395949577 +0000 UTC m=+732.215459387" watchObservedRunningTime="2026-07-13 21:41:51.398284609 +0000 UTC m=+732.217794417" Jul 13 21:41:51.413504 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.413480 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc64w"] Jul 13 21:41:51.425085 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.425065 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc64w"] Jul 13 21:41:51.427026 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.427006 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lkg8m"] Jul 13 21:41:51.427161 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.427142 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-lkg8m" podUID="984e2a17-91d0-49a3-af2a-9864912974fa" containerName="authorino" containerID="cri-o://ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0" gracePeriod=30 Jul 13 21:41:51.438188 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.438168 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64fb7789d7-jc24c"] Jul 13 21:41:51.445016 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.444995 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64fb7789d7-jc24c"] Jul 13 21:41:51.671115 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.671091 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:51.805823 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.805746 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mss7j\" (UniqueName: \"kubernetes.io/projected/984e2a17-91d0-49a3-af2a-9864912974fa-kube-api-access-mss7j\") pod \"984e2a17-91d0-49a3-af2a-9864912974fa\" (UID: \"984e2a17-91d0-49a3-af2a-9864912974fa\") " Jul 13 21:41:51.807728 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.807703 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984e2a17-91d0-49a3-af2a-9864912974fa-kube-api-access-mss7j" (OuterVolumeSpecName: "kube-api-access-mss7j") pod "984e2a17-91d0-49a3-af2a-9864912974fa" (UID: "984e2a17-91d0-49a3-af2a-9864912974fa"). InnerVolumeSpecName "kube-api-access-mss7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:41:51.907367 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:51.907342 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mss7j\" (UniqueName: \"kubernetes.io/projected/984e2a17-91d0-49a3-af2a-9864912974fa-kube-api-access-mss7j\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:41:52.351791 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.351742 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aeb449d-2145-48b7-b71c-75dfb6a04a96" path="/var/lib/kubelet/pods/0aeb449d-2145-48b7-b71c-75dfb6a04a96/volumes" Jul 13 21:41:52.352081 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.352067 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2984f302-3256-4c20-ae92-fb4b14f7c732" path="/var/lib/kubelet/pods/2984f302-3256-4c20-ae92-fb4b14f7c732/volumes" Jul 13 21:41:52.387993 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.387938 2583 generic.go:356] "Generic (PLEG): container finished" podID="984e2a17-91d0-49a3-af2a-9864912974fa" containerID="ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0" exitCode=0 Jul 13 21:41:52.388328 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.387995 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lkg8m" Jul 13 21:41:52.388328 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.388028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lkg8m" event={"ID":"984e2a17-91d0-49a3-af2a-9864912974fa","Type":"ContainerDied","Data":"ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0"} Jul 13 21:41:52.388328 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.388078 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lkg8m" event={"ID":"984e2a17-91d0-49a3-af2a-9864912974fa","Type":"ContainerDied","Data":"b4fd4c4c732db5325e9c0789c09e154fbef5b12e5d4dbd2eacb6eb5ac780dd2c"} Jul 13 21:41:52.388328 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.388104 2583 scope.go:117] "RemoveContainer" containerID="ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0" Jul 13 21:41:52.396573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.396552 2583 scope.go:117] "RemoveContainer" containerID="ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0" Jul 13 21:41:52.396846 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:41:52.396828 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0\": container with ID starting with ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0 not found: ID does not exist" containerID="ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0" Jul 13 21:41:52.396902 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.396853 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0"} err="failed to get container status \"ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0\": rpc error: code = NotFound desc = could not find container \"ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0\": container with ID starting with ce69e3b5a20817a04c550653cbc66c312cf9777beaeed767acfe2b5eaa5752f0 not found: ID does not exist" Jul 13 21:41:52.404568 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.404547 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lkg8m"] Jul 13 21:41:52.410267 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:52.410249 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lkg8m"] Jul 13 21:41:54.351250 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:54.351218 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="984e2a17-91d0-49a3-af2a-9864912974fa" path="/var/lib/kubelet/pods/984e2a17-91d0-49a3-af2a-9864912974fa/volumes" Jul 13 21:41:56.482506 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482480 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79f5876b6c-hb98n"] Jul 13 21:41:56.482935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482841 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aeb449d-2145-48b7-b71c-75dfb6a04a96" containerName="authorino" Jul 13 21:41:56.482935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482853 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aeb449d-2145-48b7-b71c-75dfb6a04a96" containerName="authorino" Jul 13 21:41:56.482935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482866 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984f302-3256-4c20-ae92-fb4b14f7c732" containerName="authorino" Jul 13 21:41:56.482935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482872 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984f302-3256-4c20-ae92-fb4b14f7c732" containerName="authorino" Jul 13 21:41:56.482935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482884 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984e2a17-91d0-49a3-af2a-9864912974fa" containerName="authorino" Jul 13 21:41:56.482935 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482890 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="984e2a17-91d0-49a3-af2a-9864912974fa" containerName="authorino" Jul 13 21:41:56.483160 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482944 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aeb449d-2145-48b7-b71c-75dfb6a04a96" containerName="authorino" Jul 13 21:41:56.483160 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482953 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2984f302-3256-4c20-ae92-fb4b14f7c732" containerName="authorino" Jul 13 21:41:56.483160 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.482959 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="984e2a17-91d0-49a3-af2a-9864912974fa" containerName="authorino" Jul 13 21:41:56.649249 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.649212 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79f5876b6c-hb98n"] Jul 13 21:41:56.649399 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.649324 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.652931 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.652908 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 21:41:56.653034 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.652912 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 21:41:56.653092 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.653044 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bj87w\"" Jul 13 21:41:56.743698 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.743635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55vlt\" (UniqueName: \"kubernetes.io/projected/f3b25904-3929-485b-9f38-51b8db7b913e-kube-api-access-55vlt\") pod \"maas-controller-79f5876b6c-hb98n\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.743698 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.743689 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3b25904-3929-485b-9f38-51b8db7b913e-webhook-cert\") pod \"maas-controller-79f5876b6c-hb98n\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.844897 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.844867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55vlt\" (UniqueName: \"kubernetes.io/projected/f3b25904-3929-485b-9f38-51b8db7b913e-kube-api-access-55vlt\") pod \"maas-controller-79f5876b6c-hb98n\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.844999 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.844912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3b25904-3929-485b-9f38-51b8db7b913e-webhook-cert\") pod \"maas-controller-79f5876b6c-hb98n\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.847144 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.847120 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3b25904-3929-485b-9f38-51b8db7b913e-webhook-cert\") pod \"maas-controller-79f5876b6c-hb98n\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.852746 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.852724 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55vlt\" (UniqueName: \"kubernetes.io/projected/f3b25904-3929-485b-9f38-51b8db7b913e-kube-api-access-55vlt\") pod \"maas-controller-79f5876b6c-hb98n\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:56.959417 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:56.959398 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:57.125868 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:57.125841 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79f5876b6c-hb98n"] Jul 13 21:41:57.127602 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:41:57.127570 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b25904_3929_485b_9f38_51b8db7b913e.slice/crio-11901e4970b913aa6c0b14286d5530c514ef167d8938e20320a9a7111cdfdefc WatchSource:0}: Error finding container 11901e4970b913aa6c0b14286d5530c514ef167d8938e20320a9a7111cdfdefc: Status 404 returned error can't find the container with id 11901e4970b913aa6c0b14286d5530c514ef167d8938e20320a9a7111cdfdefc Jul 13 21:41:57.414685 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:57.414611 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5876b6c-hb98n" event={"ID":"f3b25904-3929-485b-9f38-51b8db7b913e","Type":"ContainerStarted","Data":"11901e4970b913aa6c0b14286d5530c514ef167d8938e20320a9a7111cdfdefc"} Jul 13 21:41:59.427550 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:59.427511 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5876b6c-hb98n" event={"ID":"f3b25904-3929-485b-9f38-51b8db7b913e","Type":"ContainerStarted","Data":"8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f"} Jul 13 21:41:59.427979 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:59.427573 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:41:59.446011 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:41:59.445968 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79f5876b6c-hb98n" podStartSLOduration=1.452322632 podStartE2EDuration="3.44595335s" totalImagesPullingTime="1.993630718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:59.445009884 +0000 UTC m=+740.264519695" watchObservedRunningTime="2026-07-13 21:41:59.44595335 +0000 UTC m=+740.265463198" Jul 13 21:42:10.434583 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:10.434555 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:42:30.849197 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.849160 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b9df4b459-78qps"] Jul 13 21:42:30.855197 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.855176 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:30.858496 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.858473 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 21:42:30.858603 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.858570 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kfxsg\"" Jul 13 21:42:30.863102 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.862849 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b9df4b459-78qps"] Jul 13 21:42:30.914421 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.914398 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmwp\" (UniqueName: \"kubernetes.io/projected/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-kube-api-access-wnmwp\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:30.914525 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.914479 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-openshift-service-ca\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:30.914525 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:30.914509 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-maas-api-tls\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.015046 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.015017 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-openshift-service-ca\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.015046 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.015048 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-maas-api-tls\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.015203 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.015096 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmwp\" (UniqueName: \"kubernetes.io/projected/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-kube-api-access-wnmwp\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.015611 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.015592 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-openshift-service-ca\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.017429 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.017407 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-maas-api-tls\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.023289 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.023268 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmwp\" (UniqueName: \"kubernetes.io/projected/02d0a6cc-4924-4ba7-a5a0-3daacca69fd3-kube-api-access-wnmwp\") pod \"maas-api-5b9df4b459-78qps\" (UID: \"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3\") " pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.167523 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.167463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:31.319357 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.319326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b9df4b459-78qps"] Jul 13 21:42:31.320976 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:42:31.320946 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d0a6cc_4924_4ba7_a5a0_3daacca69fd3.slice/crio-5d8e7b49c0c94062d330d350a8a6c9f585852a405c290632d291a95bf8dad60c WatchSource:0}: Error finding container 5d8e7b49c0c94062d330d350a8a6c9f585852a405c290632d291a95bf8dad60c: Status 404 returned error can't find the container with id 5d8e7b49c0c94062d330d350a8a6c9f585852a405c290632d291a95bf8dad60c Jul 13 21:42:31.601491 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:31.601457 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" event={"ID":"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3","Type":"ContainerStarted","Data":"5d8e7b49c0c94062d330d350a8a6c9f585852a405c290632d291a95bf8dad60c"} Jul 13 21:42:33.617433 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:33.617388 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" event={"ID":"02d0a6cc-4924-4ba7-a5a0-3daacca69fd3","Type":"ContainerStarted","Data":"0d72f02f70dff834220c7534ee727508b73875a67548c21295362ee8c90f6fb2"} Jul 13 21:42:33.617825 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:33.617560 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:33.637380 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:33.637326 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" podStartSLOduration=1.893302395 podStartE2EDuration="3.637309629s" totalImagesPullingTime="1.744007234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:33.634045333 +0000 UTC m=+774.453555149" watchObservedRunningTime="2026-07-13 21:42:33.637309629 +0000 UTC m=+774.456819437" Jul 13 21:42:39.628173 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.628145 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b9df4b459-78qps" Jul 13 21:42:39.655144 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.655115 2583 generic.go:356] "Generic (PLEG): container finished" podID="cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b" containerID="b723e8760b4d4bdb3ee15937ca69d4541b3d6983ed4252743c9c57705be0a1ff" exitCode=0 Jul 13 21:42:39.655300 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.655202 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" event={"ID":"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b","Type":"ContainerDied","Data":"b723e8760b4d4bdb3ee15937ca69d4541b3d6983ed4252743c9c57705be0a1ff"} Jul 13 21:42:39.655593 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.655576 2583 scope.go:117] "RemoveContainer" containerID="b723e8760b4d4bdb3ee15937ca69d4541b3d6983ed4252743c9c57705be0a1ff" Jul 13 21:42:39.656589 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.656562 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:42:39.656672 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.656656 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:42:39.656718 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:39.656696 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:42:40.663000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:40.662968 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" event={"ID":"cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b","Type":"ContainerStarted","Data":"732dd45cc00939d45285e2a404b23595060b6a53c97939f2e96d9d1832b6fb5f"} Jul 13 21:42:41.661977 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:41.661945 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:42:41.662165 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:41.662140 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tfxd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 21:42:41.662239 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:41.662198 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" podUID="cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 21:42:42.663209 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:42.663155 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tfxd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 21:42:42.663645 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:42.663235 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" podUID="cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 21:42:43.663099 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:43.663069 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tfxd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 21:42:43.663264 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:43.663122 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" podUID="cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 21:42:44.662892 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:44.662853 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tfxd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 21:42:44.663057 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:44.662907 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" podUID="cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 21:42:45.666223 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:45.666194 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:42:45.666670 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:45.666597 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:42:45.667253 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:45.667233 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tfxd7" Jul 13 21:42:51.317142 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.317109 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf"] Jul 13 21:42:51.320112 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.320094 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.322559 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.322535 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 21:42:51.322743 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.322551 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 21:42:51.323474 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.323456 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 13 21:42:51.323580 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.323536 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b8vwz\"" Jul 13 21:42:51.328332 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.328312 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf"] Jul 13 21:42:51.388278 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.388250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.388399 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.388282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxtq\" (UniqueName: \"kubernetes.io/projected/809238fe-6b36-4077-9b43-361145c555d3-kube-api-access-mcxtq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.388399 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.388303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.388399 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.388344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.388553 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.388421 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809238fe-6b36-4077-9b43-361145c555d3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.388553 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.388488 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489325 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489301 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489446 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489446 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809238fe-6b36-4077-9b43-361145c555d3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489562 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489514 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489646 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489702 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxtq\" (UniqueName: \"kubernetes.io/projected/809238fe-6b36-4077-9b43-361145c555d3-kube-api-access-mcxtq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489767 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489750 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.489851 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489826 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.490011 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.489990 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.491561 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.491535 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809238fe-6b36-4077-9b43-361145c555d3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.491846 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.491829 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809238fe-6b36-4077-9b43-361145c555d3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.496876 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.496851 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxtq\" (UniqueName: \"kubernetes.io/projected/809238fe-6b36-4077-9b43-361145c555d3-kube-api-access-mcxtq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf\" (UID: \"809238fe-6b36-4077-9b43-361145c555d3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.632584 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.632511 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:42:51.763845 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:51.763819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf"] Jul 13 21:42:51.765598 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:42:51.765566 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809238fe_6b36_4077_9b43_361145c555d3.slice/crio-a8a6eab9d14f21c9277597fcfb3f2c0333cb248c7a4c07557b592845d8a53266 WatchSource:0}: Error finding container a8a6eab9d14f21c9277597fcfb3f2c0333cb248c7a4c07557b592845d8a53266: Status 404 returned error can't find the container with id a8a6eab9d14f21c9277597fcfb3f2c0333cb248c7a4c07557b592845d8a53266 Jul 13 21:42:52.739017 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:52.738980 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" event={"ID":"809238fe-6b36-4077-9b43-361145c555d3","Type":"ContainerStarted","Data":"a8a6eab9d14f21c9277597fcfb3f2c0333cb248c7a4c07557b592845d8a53266"} Jul 13 21:42:53.745339 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:53.745305 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" event={"ID":"809238fe-6b36-4077-9b43-361145c555d3","Type":"ContainerStarted","Data":"cc7f59999616025af768272384152ed6cf74c462437c113e2a1552b8a75ceb27"} Jul 13 21:42:53.762171 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:42:53.762119 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" podStartSLOduration=1.548891449 podStartE2EDuration="2.762103751s" totalImagesPullingTime="1.213212302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:53.761470675 +0000 UTC m=+794.580980521" watchObservedRunningTime="2026-07-13 21:42:53.762103751 +0000 UTC m=+794.581613560" Jul 13 21:43:01.633043 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:01.632969 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:43:01.633043 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:01.633030 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:43:01.644792 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:01.644749 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:43:01.798736 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:01.798710 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf" Jul 13 21:43:16.735655 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.735624 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm"] Jul 13 21:43:16.740118 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.740095 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.742461 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.742440 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 21:43:16.752910 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.747560 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm"] Jul 13 21:43:16.886452 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.886398 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.886622 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.886598 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.886679 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.886653 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.886837 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.886817 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.886888 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.886846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqrg\" (UniqueName: \"kubernetes.io/projected/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-kube-api-access-qhqrg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.886928 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.886893 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.987581 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.987581 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987540 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.987796 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987592 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.987796 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987618 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqrg\" (UniqueName: \"kubernetes.io/projected/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-kube-api-access-qhqrg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.987796 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.987796 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.988006 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.987985 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.988245 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.988221 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.988338 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.988239 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.989887 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.989867 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.990172 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.990151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:16.995404 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:16.995379 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqrg\" (UniqueName: \"kubernetes.io/projected/bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8-kube-api-access-qhqrg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm\" (UID: \"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:17.073856 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:17.073834 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:17.205622 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:17.205599 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm"] Jul 13 21:43:17.207315 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:43:17.207286 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2474b8_3eac_4c7c_a4bd_81a51afa3ca8.slice/crio-9aff9cec5839f6283e9d079e325d40c02d9ccf1b557a0833ddea4b85947be049 WatchSource:0}: Error finding container 9aff9cec5839f6283e9d079e325d40c02d9ccf1b557a0833ddea4b85947be049: Status 404 returned error can't find the container with id 9aff9cec5839f6283e9d079e325d40c02d9ccf1b557a0833ddea4b85947be049 Jul 13 21:43:17.875817 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:17.875754 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" event={"ID":"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8","Type":"ContainerStarted","Data":"43bab21ff69de33a255c6118712f32446096433cb3e2a81f0ad932e1aabc11fb"} Jul 13 21:43:17.876176 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:17.875835 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" event={"ID":"bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8","Type":"ContainerStarted","Data":"9aff9cec5839f6283e9d079e325d40c02d9ccf1b557a0833ddea4b85947be049"} Jul 13 21:43:17.893021 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:17.892973 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" podStartSLOduration=1.6983129099999998 podStartE2EDuration="1.892956739s" totalImagesPullingTime="194.643829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:43:17.890835812 +0000 UTC m=+818.710345629" watchObservedRunningTime="2026-07-13 21:43:17.892956739 +0000 UTC m=+818.712466545" Jul 13 21:43:27.074183 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:27.074147 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:27.074515 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:27.074214 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:27.085985 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:27.085961 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:43:27.940260 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:43:27.940234 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm" Jul 13 21:44:26.815303 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:26.815267 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b79df98b-9xfwr"] Jul 13 21:44:26.818839 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:26.818817 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:26.825062 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:26.825041 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b79df98b-9xfwr"] Jul 13 21:44:26.914333 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:26.914308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eba5936-4553-450f-a11a-443c14742094-tls-cert\") pod \"authorino-5b79df98b-9xfwr\" (UID: \"2eba5936-4553-450f-a11a-443c14742094\") " pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:26.914422 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:26.914370 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7jq\" (UniqueName: \"kubernetes.io/projected/2eba5936-4553-450f-a11a-443c14742094-kube-api-access-ds7jq\") pod \"authorino-5b79df98b-9xfwr\" (UID: \"2eba5936-4553-450f-a11a-443c14742094\") " pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:27.014741 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:27.014711 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7jq\" (UniqueName: \"kubernetes.io/projected/2eba5936-4553-450f-a11a-443c14742094-kube-api-access-ds7jq\") pod \"authorino-5b79df98b-9xfwr\" (UID: \"2eba5936-4553-450f-a11a-443c14742094\") " pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:27.014850 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:27.014795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eba5936-4553-450f-a11a-443c14742094-tls-cert\") pod \"authorino-5b79df98b-9xfwr\" (UID: \"2eba5936-4553-450f-a11a-443c14742094\") " pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:27.016986 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:27.016964 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eba5936-4553-450f-a11a-443c14742094-tls-cert\") pod \"authorino-5b79df98b-9xfwr\" (UID: \"2eba5936-4553-450f-a11a-443c14742094\") " pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:27.022167 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:27.022141 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7jq\" (UniqueName: \"kubernetes.io/projected/2eba5936-4553-450f-a11a-443c14742094-kube-api-access-ds7jq\") pod \"authorino-5b79df98b-9xfwr\" (UID: \"2eba5936-4553-450f-a11a-443c14742094\") " pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:27.154372 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:27.154300 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b79df98b-9xfwr" Jul 13 21:44:27.281043 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:27.281019 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b79df98b-9xfwr"] Jul 13 21:44:27.282431 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:44:27.282395 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eba5936_4553_450f_a11a_443c14742094.slice/crio-5af1f1c0323a8c4698e7b695655b8920365264425c1dfe3f492f2a99f2d72711 WatchSource:0}: Error finding container 5af1f1c0323a8c4698e7b695655b8920365264425c1dfe3f492f2a99f2d72711: Status 404 returned error can't find the container with id 5af1f1c0323a8c4698e7b695655b8920365264425c1dfe3f492f2a99f2d72711 Jul 13 21:44:28.253220 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.253121 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b79df98b-9xfwr" event={"ID":"2eba5936-4553-450f-a11a-443c14742094","Type":"ContainerStarted","Data":"4b30948e3f065a5a9078cc1e05f59e17dce97a0fe1a37370e866a3c9ce7eeece"} Jul 13 21:44:28.253220 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.253169 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b79df98b-9xfwr" event={"ID":"2eba5936-4553-450f-a11a-443c14742094","Type":"ContainerStarted","Data":"5af1f1c0323a8c4698e7b695655b8920365264425c1dfe3f492f2a99f2d72711"} Jul 13 21:44:28.276248 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.276188 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b79df98b-9xfwr" podStartSLOduration=1.609597513 podStartE2EDuration="2.276167139s" totalImagesPullingTime="666.569626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:44:28.272330185 +0000 UTC m=+889.091839994" watchObservedRunningTime="2026-07-13 21:44:28.276167139 +0000 UTC m=+889.095676948" Jul 13 21:44:28.302577 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.302510 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f7dcf78f9-594pr"] Jul 13 21:44:28.302812 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.302715 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" podUID="97be81df-c253-4af5-940b-0e413a704b54" containerName="authorino" containerID="cri-o://05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d" gracePeriod=30 Jul 13 21:44:28.549346 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.549321 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:44:28.626713 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.626679 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b98vc\" (UniqueName: \"kubernetes.io/projected/97be81df-c253-4af5-940b-0e413a704b54-kube-api-access-b98vc\") pod \"97be81df-c253-4af5-940b-0e413a704b54\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " Jul 13 21:44:28.626889 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.626815 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97be81df-c253-4af5-940b-0e413a704b54-tls-cert\") pod \"97be81df-c253-4af5-940b-0e413a704b54\" (UID: \"97be81df-c253-4af5-940b-0e413a704b54\") " Jul 13 21:44:28.628717 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.628686 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97be81df-c253-4af5-940b-0e413a704b54-kube-api-access-b98vc" (OuterVolumeSpecName: "kube-api-access-b98vc") pod "97be81df-c253-4af5-940b-0e413a704b54" (UID: "97be81df-c253-4af5-940b-0e413a704b54"). InnerVolumeSpecName "kube-api-access-b98vc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:44:28.637217 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.637186 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97be81df-c253-4af5-940b-0e413a704b54-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "97be81df-c253-4af5-940b-0e413a704b54" (UID: "97be81df-c253-4af5-940b-0e413a704b54"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:44:28.728343 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.728311 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97be81df-c253-4af5-940b-0e413a704b54-tls-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:44:28.728343 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:28.728336 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b98vc\" (UniqueName: \"kubernetes.io/projected/97be81df-c253-4af5-940b-0e413a704b54-kube-api-access-b98vc\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:44:29.260764 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.260728 2583 generic.go:356] "Generic (PLEG): container finished" podID="97be81df-c253-4af5-940b-0e413a704b54" containerID="05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d" exitCode=0 Jul 13 21:44:29.261193 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.260799 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" Jul 13 21:44:29.261193 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.260822 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" event={"ID":"97be81df-c253-4af5-940b-0e413a704b54","Type":"ContainerDied","Data":"05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d"} Jul 13 21:44:29.261193 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.260877 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f7dcf78f9-594pr" event={"ID":"97be81df-c253-4af5-940b-0e413a704b54","Type":"ContainerDied","Data":"2ca99051630a7dcd0251e19ef3cbd6365ca5c99c7c4ff29209ee1f8019ba2f4f"} Jul 13 21:44:29.261193 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.260901 2583 scope.go:117] "RemoveContainer" containerID="05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d" Jul 13 21:44:29.274122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.274097 2583 scope.go:117] "RemoveContainer" containerID="05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d" Jul 13 21:44:29.274405 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:44:29.274385 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d\": container with ID starting with 05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d not found: ID does not exist" containerID="05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d" Jul 13 21:44:29.274472 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.274417 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d"} err="failed to get container status \"05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d\": rpc error: code = NotFound desc = could not find container \"05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d\": container with ID starting with 05028d2496e9fe46c1e5343742da4db9a6f8a1300152ece220aac554b72cbd9d not found: ID does not exist" Jul 13 21:44:29.291942 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.291918 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f7dcf78f9-594pr"] Jul 13 21:44:29.296230 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:29.296206 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7f7dcf78f9-594pr"] Jul 13 21:44:30.352619 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:30.352587 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97be81df-c253-4af5-940b-0e413a704b54" path="/var/lib/kubelet/pods/97be81df-c253-4af5-940b-0e413a704b54/volumes" Jul 13 21:44:40.445893 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:40.445821 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:44:40.448144 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:44:40.448125 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:45:38.287768 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.287736 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79f5876b6c-hb98n"] Jul 13 21:45:38.288371 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.288040 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79f5876b6c-hb98n" podUID="f3b25904-3929-485b-9f38-51b8db7b913e" containerName="manager" containerID="cri-o://8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f" gracePeriod=30 Jul 13 21:45:38.535894 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.535872 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:45:38.619690 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.619603 2583 generic.go:356] "Generic (PLEG): container finished" podID="f3b25904-3929-485b-9f38-51b8db7b913e" containerID="8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f" exitCode=0 Jul 13 21:45:38.619690 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.619656 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5876b6c-hb98n" Jul 13 21:45:38.619690 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.619672 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5876b6c-hb98n" event={"ID":"f3b25904-3929-485b-9f38-51b8db7b913e","Type":"ContainerDied","Data":"8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f"} Jul 13 21:45:38.619915 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.619746 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5876b6c-hb98n" event={"ID":"f3b25904-3929-485b-9f38-51b8db7b913e","Type":"ContainerDied","Data":"11901e4970b913aa6c0b14286d5530c514ef167d8938e20320a9a7111cdfdefc"} Jul 13 21:45:38.619915 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.619782 2583 scope.go:117] "RemoveContainer" containerID="8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f" Jul 13 21:45:38.628091 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.628071 2583 scope.go:117] "RemoveContainer" containerID="8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f" Jul 13 21:45:38.628313 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:45:38.628289 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f\": container with ID starting with 8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f not found: ID does not exist" containerID="8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f" Jul 13 21:45:38.628404 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.628317 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f"} err="failed to get container status \"8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f\": rpc error: code = NotFound desc = could not find container \"8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f\": container with ID starting with 8ba852627a2a4e20b9e78358f28d79891ca4d1077bb7e9b6b4276cd6ebabcc9f not found: ID does not exist" Jul 13 21:45:38.682706 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.682683 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55vlt\" (UniqueName: \"kubernetes.io/projected/f3b25904-3929-485b-9f38-51b8db7b913e-kube-api-access-55vlt\") pod \"f3b25904-3929-485b-9f38-51b8db7b913e\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " Jul 13 21:45:38.682823 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.682749 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3b25904-3929-485b-9f38-51b8db7b913e-webhook-cert\") pod \"f3b25904-3929-485b-9f38-51b8db7b913e\" (UID: \"f3b25904-3929-485b-9f38-51b8db7b913e\") " Jul 13 21:45:38.684728 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.684709 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3b25904-3929-485b-9f38-51b8db7b913e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f3b25904-3929-485b-9f38-51b8db7b913e" (UID: "f3b25904-3929-485b-9f38-51b8db7b913e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:45:38.684858 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.684837 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3b25904-3929-485b-9f38-51b8db7b913e-kube-api-access-55vlt" (OuterVolumeSpecName: "kube-api-access-55vlt") pod "f3b25904-3929-485b-9f38-51b8db7b913e" (UID: "f3b25904-3929-485b-9f38-51b8db7b913e"). InnerVolumeSpecName "kube-api-access-55vlt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:45:38.783484 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.783461 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55vlt\" (UniqueName: \"kubernetes.io/projected/f3b25904-3929-485b-9f38-51b8db7b913e-kube-api-access-55vlt\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:45:38.783484 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.783483 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3b25904-3929-485b-9f38-51b8db7b913e-webhook-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:45:38.942539 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.942510 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79f5876b6c-hb98n"] Jul 13 21:45:38.946683 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:38.946661 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79f5876b6c-hb98n"] Jul 13 21:45:40.353047 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:45:40.353019 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3b25904-3929-485b-9f38-51b8db7b913e" path="/var/lib/kubelet/pods/f3b25904-3929-485b-9f38-51b8db7b913e/volumes" Jul 13 21:49:40.480238 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:49:40.480209 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:49:40.483560 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:49:40.483538 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:54:40.521363 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:54:40.521255 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:54:40.526425 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:54:40.526406 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:57:49.032238 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032207 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv"] Jul 13 21:57:49.032615 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032586 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b25904-3929-485b-9f38-51b8db7b913e" containerName="manager" Jul 13 21:57:49.032615 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032598 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b25904-3929-485b-9f38-51b8db7b913e" containerName="manager" Jul 13 21:57:49.032692 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032618 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97be81df-c253-4af5-940b-0e413a704b54" containerName="authorino" Jul 13 21:57:49.032692 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032624 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="97be81df-c253-4af5-940b-0e413a704b54" containerName="authorino" Jul 13 21:57:49.032692 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032679 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b25904-3929-485b-9f38-51b8db7b913e" containerName="manager" Jul 13 21:57:49.032692 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.032687 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="97be81df-c253-4af5-940b-0e413a704b54" containerName="authorino" Jul 13 21:57:49.035883 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.035863 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.040630 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.040604 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-56bf06f8-openshift-default-dockercfg-5v27m\"" Jul 13 21:57:49.059659 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.059633 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv"] Jul 13 21:57:49.075737 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075713 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-podinfo\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.075851 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075741 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e763197-115f-47de-b1a1-d773e9f6ce7b-istiod-ca-cert\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.075851 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075804 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-envoy\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.075851 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-credential-socket\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.075851 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-socket\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.075989 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075935 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44q9p\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-kube-api-access-44q9p\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.075989 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.075976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-token\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.076123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.076003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-certs\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.076123 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.076024 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-data\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177365 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177328 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-credential-socket\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177365 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-socket\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177417 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44q9p\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-kube-api-access-44q9p\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-token\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-certs\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-data\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-podinfo\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177606 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e763197-115f-47de-b1a1-d773e9f6ce7b-istiod-ca-cert\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177950 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-envoy\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177950 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177837 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-credential-socket\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.177950 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177888 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-certs\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.178120 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.177965 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-data\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.178120 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.178095 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-socket\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.178575 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.178553 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e763197-115f-47de-b1a1-d773e9f6ce7b-istiod-ca-cert\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.180041 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.180019 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-envoy\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.180166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.180149 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-podinfo\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.185626 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.185597 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44q9p\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-kube-api-access-44q9p\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.185729 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.185651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-token\") pod \"e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.345793 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.345714 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:49.499793 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:57:49.499744 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e763197_115f_47de_b1a1_d773e9f6ce7b.slice/crio-704ce2234ec27ba4532b2f2cfcbc6a3a07011a9717e7413657902bf316edd559 WatchSource:0}: Error finding container 704ce2234ec27ba4532b2f2cfcbc6a3a07011a9717e7413657902bf316edd559: Status 404 returned error can't find the container with id 704ce2234ec27ba4532b2f2cfcbc6a3a07011a9717e7413657902bf316edd559 Jul 13 21:57:49.500867 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.500840 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv"] Jul 13 21:57:49.501843 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.501811 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:57:49.501925 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.501874 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:57:49.501925 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:49.501905 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:57:50.507394 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:50.507358 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" event={"ID":"6e763197-115f-47de-b1a1-d773e9f6ce7b","Type":"ContainerStarted","Data":"9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b"} Jul 13 21:57:50.507877 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:50.507415 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" event={"ID":"6e763197-115f-47de-b1a1-d773e9f6ce7b","Type":"ContainerStarted","Data":"704ce2234ec27ba4532b2f2cfcbc6a3a07011a9717e7413657902bf316edd559"} Jul 13 21:57:50.530510 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:50.530467 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" podStartSLOduration=2.530450631 podStartE2EDuration="2.530450631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:57:50.52957962 +0000 UTC m=+1691.349089428" watchObservedRunningTime="2026-07-13 21:57:50.530450631 +0000 UTC m=+1691.349960437" Jul 13 21:57:51.345922 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:51.345885 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:51.351011 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:51.350985 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:51.513476 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:51.513438 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:57:51.514388 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:57:51.514370 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:58:04.596630 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:04.596564 2583 generic.go:356] "Generic (PLEG): container finished" podID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerID="9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b" exitCode=0 Jul 13 21:58:04.596999 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:04.596643 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" event={"ID":"6e763197-115f-47de-b1a1-d773e9f6ce7b","Type":"ContainerDied","Data":"9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b"} Jul 13 21:58:04.597064 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:04.597046 2583 scope.go:117] "RemoveContainer" containerID="9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b" Jul 13 21:58:04.598082 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:04.598041 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:58:04.598190 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:04.598126 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:58:04.598190 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:04.598157 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:58:05.604297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:05.604262 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" event={"ID":"6e763197-115f-47de-b1a1-d773e9f6ce7b","Type":"ContainerStarted","Data":"d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618"} Jul 13 21:58:06.346589 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:06.346534 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:58:06.347728 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:06.347697 2583 patch_prober.go:28] interesting pod/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 21:58:06.347879 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:06.347756 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 21:58:06.513713 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:06.513681 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:58:07.346937 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:07.346888 2583 patch_prober.go:28] interesting pod/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 21:58:07.347333 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:07.346956 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 21:58:08.346166 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:08.346128 2583 patch_prober.go:28] interesting pod/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 13 21:58:08.346330 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:08.346190 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 13 21:58:09.349929 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:09.349900 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:58:09.351035 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:58:09.351015 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:59:10.679646 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:10.679613 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv"] Jul 13 21:59:10.680129 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:10.679833 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" containerID="cri-o://d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618" gracePeriod=30 Jul 13 21:59:11.487090 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.487058 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9"] Jul 13 21:59:11.490376 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.490360 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.495349 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.495329 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7e874d59-gw-openshift-default-dockercfg-w2lxg\"" Jul 13 21:59:11.518387 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.518365 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9"] Jul 13 21:59:11.535994 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.535976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-credential-socket\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536092 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536012 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-socket\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536092 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536029 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dmfq\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-kube-api-access-9dmfq\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536092 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-certs\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536198 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536097 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-istio-token\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536198 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-envoy\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536261 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536210 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-data\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536261 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19ace19e-1155-4c68-b1f8-630385c4b737-istiod-ca-cert\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.536261 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.536256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19ace19e-1155-4c68-b1f8-630385c4b737-istio-podinfo\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637327 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-data\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637327 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637328 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19ace19e-1155-4c68-b1f8-630385c4b737-istiod-ca-cert\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19ace19e-1155-4c68-b1f8-630385c4b737-istio-podinfo\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-credential-socket\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637489 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-socket\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmfq\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-kube-api-access-9dmfq\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637573 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-certs\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637857 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637587 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-istio-token\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637857 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637636 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-envoy\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637965 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637869 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-data\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.637965 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.637948 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-credential-socket\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.638169 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.638147 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-socket\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.638232 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.638186 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19ace19e-1155-4c68-b1f8-630385c4b737-istiod-ca-cert\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.638232 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.638210 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-certs\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.639784 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.639756 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-envoy\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.640016 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.639997 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19ace19e-1155-4c68-b1f8-630385c4b737-istio-podinfo\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.654846 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.654824 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmfq\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-kube-api-access-9dmfq\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.664133 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.664114 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-istio-token\") pod \"e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.799834 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.799748 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:11.949917 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.949882 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9"] Jul 13 21:59:11.950425 ip-10-0-133-19 kubenswrapper[2583]: W0713 21:59:11.950401 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ace19e_1155_4c68_b1f8_630385c4b737.slice/crio-33a614476e65870890a1d97c10005e18c03c958924442436e3270fcf4d98a54f WatchSource:0}: Error finding container 33a614476e65870890a1d97c10005e18c03c958924442436e3270fcf4d98a54f: Status 404 returned error can't find the container with id 33a614476e65870890a1d97c10005e18c03c958924442436e3270fcf4d98a54f Jul 13 21:59:11.952349 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.952315 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:59:11.952438 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.952392 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:59:11.952551 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:11.952436 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:59:12.957187 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:12.957146 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" event={"ID":"19ace19e-1155-4c68-b1f8-630385c4b737","Type":"ContainerStarted","Data":"605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3"} Jul 13 21:59:12.957556 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:12.957213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" event={"ID":"19ace19e-1155-4c68-b1f8-630385c4b737","Type":"ContainerStarted","Data":"33a614476e65870890a1d97c10005e18c03c958924442436e3270fcf4d98a54f"} Jul 13 21:59:13.800962 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:13.800928 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:13.805651 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:13.805630 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:13.847649 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:13.847587 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podStartSLOduration=2.847565499 podStartE2EDuration="2.847565499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:13.008023304 +0000 UTC m=+1773.827533111" watchObservedRunningTime="2026-07-13 21:59:13.847565499 +0000 UTC m=+1774.667075310" Jul 13 21:59:13.963299 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:13.963268 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:13.964246 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:13.964226 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:15.925401 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.925378 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:59:15.977279 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.977207 2583 generic.go:356] "Generic (PLEG): container finished" podID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerID="d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618" exitCode=0 Jul 13 21:59:15.977279 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.977255 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" Jul 13 21:59:15.977432 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.977290 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" event={"ID":"6e763197-115f-47de-b1a1-d773e9f6ce7b","Type":"ContainerDied","Data":"d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618"} Jul 13 21:59:15.977432 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.977370 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv" event={"ID":"6e763197-115f-47de-b1a1-d773e9f6ce7b","Type":"ContainerDied","Data":"704ce2234ec27ba4532b2f2cfcbc6a3a07011a9717e7413657902bf316edd559"} Jul 13 21:59:15.977432 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.977400 2583 scope.go:117] "RemoveContainer" containerID="d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618" Jul 13 21:59:15.986492 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.986462 2583 scope.go:117] "RemoveContainer" containerID="9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b" Jul 13 21:59:15.995586 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.995567 2583 scope.go:117] "RemoveContainer" containerID="d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618" Jul 13 21:59:15.995881 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:59:15.995861 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618\": container with ID starting with d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618 not found: ID does not exist" containerID="d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618" Jul 13 21:59:15.995960 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.995889 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618"} err="failed to get container status \"d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618\": rpc error: code = NotFound desc = could not find container \"d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618\": container with ID starting with d6ee13a4511e0d8db8c48b1d792cedd983904bbc9ab006c4771b6b41592c4618 not found: ID does not exist" Jul 13 21:59:15.995960 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.995905 2583 scope.go:117] "RemoveContainer" containerID="9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b" Jul 13 21:59:15.996174 ip-10-0-133-19 kubenswrapper[2583]: E0713 21:59:15.996151 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b\": container with ID starting with 9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b not found: ID does not exist" containerID="9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b" Jul 13 21:59:15.996216 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:15.996184 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b"} err="failed to get container status \"9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b\": rpc error: code = NotFound desc = could not find container \"9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b\": container with ID starting with 9fd551c8a8ac14041e1da57e8232c67543d114f718ad40872739b071caed4d4b not found: ID does not exist" Jul 13 21:59:16.076622 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076597 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-certs\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.076753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076637 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44q9p\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-kube-api-access-44q9p\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.076753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076667 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-socket\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.076753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076701 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-credential-socket\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.076753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076728 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-data\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076759 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-token\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076818 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e763197-115f-47de-b1a1-d773e9f6ce7b-istiod-ca-cert\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076872 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-podinfo\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076878 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:59:16.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076925 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-envoy\") pod \"6e763197-115f-47de-b1a1-d773e9f6ce7b\" (UID: \"6e763197-115f-47de-b1a1-d773e9f6ce7b\") " Jul 13 21:59:16.077269 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.076998 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:59:16.077269 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077010 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:59:16.077269 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077197 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e763197-115f-47de-b1a1-d773e9f6ce7b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:59:16.077613 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077590 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.077694 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077616 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.077694 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077616 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:59:16.077694 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077631 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.077694 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.077646 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e763197-115f-47de-b1a1-d773e9f6ce7b-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.078950 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.078928 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-kube-api-access-44q9p" (OuterVolumeSpecName: "kube-api-access-44q9p") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "kube-api-access-44q9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:59:16.079073 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.079056 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 21:59:16.079122 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.079085 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:59:16.079297 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.079281 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e763197-115f-47de-b1a1-d773e9f6ce7b" (UID: "6e763197-115f-47de-b1a1-d773e9f6ce7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:59:16.178190 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.178166 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.178190 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.178187 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.178392 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.178195 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.178392 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.178204 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e763197-115f-47de-b1a1-d773e9f6ce7b-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.178392 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.178211 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44q9p\" (UniqueName: \"kubernetes.io/projected/6e763197-115f-47de-b1a1-d773e9f6ce7b-kube-api-access-44q9p\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 21:59:16.314011 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.313980 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv"] Jul 13 21:59:16.316149 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.316130 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-56bf06f8-openshift-default-54cd6b6c77-sfwrv"] Jul 13 21:59:16.352859 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:16.352834 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" path="/var/lib/kubelet/pods/6e763197-115f-47de-b1a1-d773e9f6ce7b/volumes" Jul 13 21:59:28.965023 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:28.964994 2583 patch_prober.go:28] interesting pod/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 21:59:28.965401 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:28.965062 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 21:59:29.058592 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:29.058566 2583 generic.go:356] "Generic (PLEG): container finished" podID="19ace19e-1155-4c68-b1f8-630385c4b737" containerID="605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3" exitCode=0 Jul 13 21:59:29.058753 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:29.058640 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" event={"ID":"19ace19e-1155-4c68-b1f8-630385c4b737","Type":"ContainerDied","Data":"605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3"} Jul 13 21:59:29.059000 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:29.058984 2583 scope.go:117] "RemoveContainer" containerID="605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3" Jul 13 21:59:29.059940 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:29.059908 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:59:29.060059 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:29.059978 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:59:29.060059 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:29.060013 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 21:59:30.066387 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:30.066356 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" event={"ID":"19ace19e-1155-4c68-b1f8-630385c4b737","Type":"ContainerStarted","Data":"4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57"} Jul 13 21:59:30.800224 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:30.800176 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:30.800447 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:30.800414 2583 patch_prober.go:28] interesting pod/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 21:59:30.800583 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:30.800464 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 21:59:31.800444 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:31.800371 2583 patch_prober.go:28] interesting pod/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 21:59:31.800444 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:31.800425 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 21:59:32.800191 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:32.800156 2583 patch_prober.go:28] interesting pod/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 21:59:32.800362 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:32.800216 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 21:59:33.804360 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:33.804323 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:33.804730 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:33.804602 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:33.805206 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:33.805187 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 21:59:40.559284 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:40.559255 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 21:59:40.565831 ip-10-0-133-19 kubenswrapper[2583]: I0713 21:59:40.565813 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:00:17.282854 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:17.282820 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9"] Jul 13 22:00:17.283383 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:17.283011 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" containerID="cri-o://4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57" gracePeriod=30 Jul 13 22:00:18.807079 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:18.807042 2583 patch_prober.go:28] interesting pod/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:00:18.807426 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:18.807105 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:00:22.541515 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.541491 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 22:00:22.621475 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621446 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19ace19e-1155-4c68-b1f8-630385c4b737-istiod-ca-cert\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621637 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621515 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-credential-socket\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621637 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621560 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-istio-token\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621637 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621608 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19ace19e-1155-4c68-b1f8-630385c4b737-istio-podinfo\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621639 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-certs\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621801 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-socket\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621937 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621861 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-data\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.621937 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621876 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:00:22.622052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621895 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-envoy\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.622052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621953 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19ace19e-1155-4c68-b1f8-630385c4b737-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:00:22.622052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621984 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dmfq\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-kube-api-access-9dmfq\") pod \"19ace19e-1155-4c68-b1f8-630385c4b737\" (UID: \"19ace19e-1155-4c68-b1f8-630385c4b737\") " Jul 13 22:00:22.622052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.621988 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:00:22.622279 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.622171 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:00:22.622337 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.622315 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.622392 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.622336 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.622392 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.622350 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.622392 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.622363 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19ace19e-1155-4c68-b1f8-630385c4b737-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.622913 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.622886 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-data" (OuterVolumeSpecName: "istio-data") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:00:22.623980 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.623947 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19ace19e-1155-4c68-b1f8-630385c4b737-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:00:22.624086 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.624002 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:00:22.624086 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.624007 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-istio-token" (OuterVolumeSpecName: "istio-token") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:00:22.624368 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.624346 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-kube-api-access-9dmfq" (OuterVolumeSpecName: "kube-api-access-9dmfq") pod "19ace19e-1155-4c68-b1f8-630385c4b737" (UID: "19ace19e-1155-4c68-b1f8-630385c4b737"). InnerVolumeSpecName "kube-api-access-9dmfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:00:22.723112 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.723078 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.723112 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.723105 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19ace19e-1155-4c68-b1f8-630385c4b737-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.723112 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.723115 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dmfq\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-kube-api-access-9dmfq\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.723112 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.723124 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19ace19e-1155-4c68-b1f8-630385c4b737-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:22.723462 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:22.723133 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19ace19e-1155-4c68-b1f8-630385c4b737-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:00:23.362172 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.362144 2583 generic.go:356] "Generic (PLEG): container finished" podID="19ace19e-1155-4c68-b1f8-630385c4b737" containerID="4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57" exitCode=0 Jul 13 22:00:23.362326 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.362191 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" Jul 13 22:00:23.362326 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.362205 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" event={"ID":"19ace19e-1155-4c68-b1f8-630385c4b737","Type":"ContainerDied","Data":"4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57"} Jul 13 22:00:23.362326 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.362243 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9" event={"ID":"19ace19e-1155-4c68-b1f8-630385c4b737","Type":"ContainerDied","Data":"33a614476e65870890a1d97c10005e18c03c958924442436e3270fcf4d98a54f"} Jul 13 22:00:23.362326 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.362265 2583 scope.go:117] "RemoveContainer" containerID="4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57" Jul 13 22:00:23.371218 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.371201 2583 scope.go:117] "RemoveContainer" containerID="605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3" Jul 13 22:00:23.379043 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.379028 2583 scope.go:117] "RemoveContainer" containerID="4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57" Jul 13 22:00:23.379272 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:00:23.379254 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57\": container with ID starting with 4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57 not found: ID does not exist" containerID="4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57" Jul 13 22:00:23.379317 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.379279 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57"} err="failed to get container status \"4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57\": rpc error: code = NotFound desc = could not find container \"4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57\": container with ID starting with 4a9339b0c4c26db0fd2ccb9c84dd40a2a16f53f356f16a559c591be4b2cf4a57 not found: ID does not exist" Jul 13 22:00:23.379317 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.379295 2583 scope.go:117] "RemoveContainer" containerID="605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3" Jul 13 22:00:23.379498 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:00:23.379483 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3\": container with ID starting with 605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3 not found: ID does not exist" containerID="605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3" Jul 13 22:00:23.379538 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.379503 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3"} err="failed to get container status \"605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3\": rpc error: code = NotFound desc = could not find container \"605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3\": container with ID starting with 605ab0eb4b13befe81894ddae698d5070340bbf360489aa68de1328243d6d1d3 not found: ID does not exist" Jul 13 22:00:23.383700 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.383673 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9"] Jul 13 22:00:23.386786 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.386754 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7e874d59-gw-openshift-default-557cb57d9f-4cdd9"] Jul 13 22:00:23.671474 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671446 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws"] Jul 13 22:00:23.671845 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671831 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671846 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671860 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671865 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671872 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671878 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671892 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" Jul 13 22:00:23.671904 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671899 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" Jul 13 22:00:23.672123 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671968 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" Jul 13 22:00:23.672123 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671976 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e763197-115f-47de-b1a1-d773e9f6ce7b" containerName="istio-proxy" Jul 13 22:00:23.672123 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671983 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" Jul 13 22:00:23.672123 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.671989 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" containerName="istio-proxy" Jul 13 22:00:23.676523 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.676500 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.679537 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.679510 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-30bfd308-openshift-default-dockercfg-csxmt\"" Jul 13 22:00:23.689367 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.689346 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws"] Jul 13 22:00:23.832523 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832488 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-envoy\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832680 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832538 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctmc9\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-kube-api-access-ctmc9\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832680 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832604 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-token\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832680 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832664 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a025a757-1b8f-4b0a-9387-44daeb63bd69-istiod-ca-cert\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832697 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-credential-socket\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832734 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-certs\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-podinfo\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.832853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832842 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-data\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.833040 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.832942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-socket\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934331 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-credential-socket\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934517 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934349 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-certs\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934517 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-podinfo\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934517 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-data\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934517 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934442 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-socket\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934517 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-envoy\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934866 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctmc9\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-kube-api-access-ctmc9\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.934866 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934860 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-data\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.935008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934749 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-credential-socket\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.935008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934842 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-certs\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.935008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934926 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-socket\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.935008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-token\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.935008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.934989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a025a757-1b8f-4b0a-9387-44daeb63bd69-istiod-ca-cert\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.935483 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.935458 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a025a757-1b8f-4b0a-9387-44daeb63bd69-istiod-ca-cert\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.936710 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.936689 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-envoy\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.937045 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.937025 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-podinfo\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.942990 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.942967 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctmc9\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-kube-api-access-ctmc9\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.943911 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.943892 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-token\") pod \"e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:23.989211 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:23.989176 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:24.128360 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.128334 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws"] Jul 13 22:00:24.130285 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:00:24.130246 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda025a757_1b8f_4b0a_9387_44daeb63bd69.slice/crio-a61c56aaaeb35138366afd0b20188745623f0928143f2094638cddd03ec27c7f WatchSource:0}: Error finding container a61c56aaaeb35138366afd0b20188745623f0928143f2094638cddd03ec27c7f: Status 404 returned error can't find the container with id a61c56aaaeb35138366afd0b20188745623f0928143f2094638cddd03ec27c7f Jul 13 22:00:24.132423 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.132381 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:00:24.132522 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.132502 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:00:24.132570 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.132554 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:00:24.352980 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.352949 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19ace19e-1155-4c68-b1f8-630385c4b737" path="/var/lib/kubelet/pods/19ace19e-1155-4c68-b1f8-630385c4b737/volumes" Jul 13 22:00:24.369913 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.369882 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" event={"ID":"a025a757-1b8f-4b0a-9387-44daeb63bd69","Type":"ContainerStarted","Data":"544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f"} Jul 13 22:00:24.370051 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.369929 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" event={"ID":"a025a757-1b8f-4b0a-9387-44daeb63bd69","Type":"ContainerStarted","Data":"a61c56aaaeb35138366afd0b20188745623f0928143f2094638cddd03ec27c7f"} Jul 13 22:00:24.391753 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.391696 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podStartSLOduration=1.391676653 podStartE2EDuration="1.391676653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:24.389616496 +0000 UTC m=+1845.209126326" watchObservedRunningTime="2026-07-13 22:00:24.391676653 +0000 UTC m=+1845.211186462" Jul 13 22:00:24.989602 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:24.989571 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:25.994696 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:25.994667 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:26.380196 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:26.380119 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:26.381213 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:26.381193 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:41.381838 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.381757 2583 patch_prober.go:28] interesting pod/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 13 22:00:41.381838 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.381830 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 13 22:00:41.465330 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.465300 2583 generic.go:356] "Generic (PLEG): container finished" podID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerID="544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f" exitCode=0 Jul 13 22:00:41.465455 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.465370 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" event={"ID":"a025a757-1b8f-4b0a-9387-44daeb63bd69","Type":"ContainerDied","Data":"544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f"} Jul 13 22:00:41.465757 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.465742 2583 scope.go:117] "RemoveContainer" containerID="544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f" Jul 13 22:00:41.466826 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.466797 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:00:41.466897 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.466878 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:00:41.466942 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:41.466927 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:00:42.472602 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:42.472563 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" event={"ID":"a025a757-1b8f-4b0a-9387-44daeb63bd69","Type":"ContainerStarted","Data":"039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8"} Jul 13 22:00:42.989322 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:42.989268 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:42.989938 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:42.989844 2583 patch_prober.go:28] interesting pod/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 13 22:00:42.990152 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:42.990128 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 13 22:00:43.990155 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:43.990108 2583 patch_prober.go:28] interesting pod/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 13 22:00:43.990658 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:43.990213 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 13 22:00:44.990129 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:44.990089 2583 patch_prober.go:28] interesting pod/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 13 22:00:44.990425 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:44.990395 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 13 22:00:45.990224 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:45.990181 2583 patch_prober.go:28] interesting pod/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 13 22:00:45.990421 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:45.990259 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 13 22:00:46.994256 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:46.994227 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:46.994707 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:46.994688 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:00:46.995393 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:00:46.995372 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:01:46.082668 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.082632 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws"] Jul 13 22:01:46.083182 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.082868 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" containerID="cri-o://039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8" gracePeriod=30 Jul 13 22:01:46.403351 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.403274 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t"] Jul 13 22:01:46.406931 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.406911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.410549 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.410525 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-628bb6aa-openshift-default-dockercfg-8dvhc\"" Jul 13 22:01:46.422442 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.422421 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t"] Jul 13 22:01:46.533119 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533077 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-envoy\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533282 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-credential-socket\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533282 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533177 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsr69\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-kube-api-access-lsr69\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533282 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-data\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533282 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-token\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533480 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-socket\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533480 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533310 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e767d066-973a-484c-9385-c20c7d9c7d2d-istiod-ca-cert\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533480 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533365 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-certs\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.533480 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.533457 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-podinfo\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634120 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634079 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-certs\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-podinfo\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634170 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-envoy\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-credential-socket\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634217 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsr69\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-kube-api-access-lsr69\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-data\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634273 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-token\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634311 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-socket\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634712 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e767d066-973a-484c-9385-c20c7d9c7d2d-istiod-ca-cert\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634712 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634547 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-certs\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634861 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634807 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-credential-socket\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634919 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634875 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-data\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.634961 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.634915 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-socket\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.635138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.635120 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e767d066-973a-484c-9385-c20c7d9c7d2d-istiod-ca-cert\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.637058 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.637036 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-envoy\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.637251 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.637231 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-podinfo\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.642491 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.642471 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-token\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.642889 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.642870 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsr69\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-kube-api-access-lsr69\") pod \"e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.719530 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.719500 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:46.851638 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.851564 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t"] Jul 13 22:01:46.855218 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:01:46.855178 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode767d066_973a_484c_9385_c20c7d9c7d2d.slice/crio-79f2aec0fb84645d91604483317ff8133fe1773f106eb43d479d5c467266e2ba WatchSource:0}: Error finding container 79f2aec0fb84645d91604483317ff8133fe1773f106eb43d479d5c467266e2ba: Status 404 returned error can't find the container with id 79f2aec0fb84645d91604483317ff8133fe1773f106eb43d479d5c467266e2ba Jul 13 22:01:46.857331 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.857299 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:01:46.857433 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.857373 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:01:46.857433 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.857417 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:01:46.997143 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.997071 2583 patch_prober.go:28] interesting pod/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:01:46.997143 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:46.997123 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:01:47.837705 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:47.837665 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" event={"ID":"e767d066-973a-484c-9385-c20c7d9c7d2d","Type":"ContainerStarted","Data":"f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6"} Jul 13 22:01:47.838145 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:47.837716 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" event={"ID":"e767d066-973a-484c-9385-c20c7d9c7d2d","Type":"ContainerStarted","Data":"79f2aec0fb84645d91604483317ff8133fe1773f106eb43d479d5c467266e2ba"} Jul 13 22:01:47.868188 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:47.868142 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podStartSLOduration=1.868123737 podStartE2EDuration="1.868123737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:47.866116445 +0000 UTC m=+1928.685626282" watchObservedRunningTime="2026-07-13 22:01:47.868123737 +0000 UTC m=+1928.687633547" Jul 13 22:01:48.720709 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:48.720672 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:48.725237 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:48.725212 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:48.843288 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:48.843249 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:48.844156 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:48.844133 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:01:51.337947 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.337920 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:01:51.370838 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a025a757-1b8f-4b0a-9387-44daeb63bd69-istiod-ca-cert\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.370838 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370813 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-token\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370841 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-credential-socket\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370867 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-socket\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370899 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-data\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370964 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-envoy\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371008 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.370997 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-certs\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371053 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-podinfo\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371083 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctmc9\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-kube-api-access-ctmc9\") pod \"a025a757-1b8f-4b0a-9387-44daeb63bd69\" (UID: \"a025a757-1b8f-4b0a-9387-44daeb63bd69\") " Jul 13 22:01:51.371253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371148 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:01:51.371253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371161 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:01:51.371253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371203 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a025a757-1b8f-4b0a-9387-44daeb63bd69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:51.371529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371360 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:01:51.371529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371427 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.371529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371446 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.371529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371461 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a025a757-1b8f-4b0a-9387-44daeb63bd69-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.371529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371473 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.371846 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.371813 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-data" (OuterVolumeSpecName: "istio-data") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:01:51.373548 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.373507 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-token" (OuterVolumeSpecName: "istio-token") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:01:51.373641 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.373577 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:01:51.373641 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.373588 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:01:51.373641 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.373607 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-kube-api-access-ctmc9" (OuterVolumeSpecName: "kube-api-access-ctmc9") pod "a025a757-1b8f-4b0a-9387-44daeb63bd69" (UID: "a025a757-1b8f-4b0a-9387-44daeb63bd69"). InnerVolumeSpecName "kube-api-access-ctmc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:01:51.472631 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.472595 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.472631 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.472627 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctmc9\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-kube-api-access-ctmc9\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.472631 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.472638 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.472868 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.472646 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.472868 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.472655 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a025a757-1b8f-4b0a-9387-44daeb63bd69-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:01:51.860185 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.860150 2583 generic.go:356] "Generic (PLEG): container finished" podID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerID="039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8" exitCode=0 Jul 13 22:01:51.860334 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.860197 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" Jul 13 22:01:51.860334 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.860192 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" event={"ID":"a025a757-1b8f-4b0a-9387-44daeb63bd69","Type":"ContainerDied","Data":"039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8"} Jul 13 22:01:51.860334 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.860317 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws" event={"ID":"a025a757-1b8f-4b0a-9387-44daeb63bd69","Type":"ContainerDied","Data":"a61c56aaaeb35138366afd0b20188745623f0928143f2094638cddd03ec27c7f"} Jul 13 22:01:51.860474 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.860348 2583 scope.go:117] "RemoveContainer" containerID="039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8" Jul 13 22:01:51.869869 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.869842 2583 scope.go:117] "RemoveContainer" containerID="544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f" Jul 13 22:01:51.878030 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.878013 2583 scope.go:117] "RemoveContainer" containerID="039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8" Jul 13 22:01:51.878285 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:01:51.878267 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8\": container with ID starting with 039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8 not found: ID does not exist" containerID="039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8" Jul 13 22:01:51.878374 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.878298 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8"} err="failed to get container status \"039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8\": rpc error: code = NotFound desc = could not find container \"039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8\": container with ID starting with 039905f89535ce1425c37409f6577d7edc98e0d44beba946903f7c768a8f81a8 not found: ID does not exist" Jul 13 22:01:51.878374 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.878320 2583 scope.go:117] "RemoveContainer" containerID="544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f" Jul 13 22:01:51.878595 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:01:51.878574 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f\": container with ID starting with 544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f not found: ID does not exist" containerID="544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f" Jul 13 22:01:51.878631 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.878602 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f"} err="failed to get container status \"544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f\": rpc error: code = NotFound desc = could not find container \"544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f\": container with ID starting with 544af978385c2b874e5149c0b77b40c98f0681656d83c0d20fd30d9488942c6f not found: ID does not exist" Jul 13 22:01:51.882786 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.882751 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws"] Jul 13 22:01:51.887119 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:51.887099 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-30bfd308-openshift-default-84d5d5d6f9-s2rws"] Jul 13 22:01:52.352874 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:01:52.352844 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" path="/var/lib/kubelet/pods/a025a757-1b8f-4b0a-9387-44daeb63bd69/volumes" Jul 13 22:02:02.937795 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:02.937739 2583 generic.go:356] "Generic (PLEG): container finished" podID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerID="f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6" exitCode=0 Jul 13 22:02:02.938240 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:02.937815 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" event={"ID":"e767d066-973a-484c-9385-c20c7d9c7d2d","Type":"ContainerDied","Data":"f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6"} Jul 13 22:02:02.938240 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:02.938212 2583 scope.go:117] "RemoveContainer" containerID="f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6" Jul 13 22:02:02.939304 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:02.939274 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:02:02.939376 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:02.939342 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:02:02.939423 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:02.939375 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:02:03.843229 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:03.843201 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:02:03.948930 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:03.948890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" event={"ID":"e767d066-973a-484c-9385-c20c7d9c7d2d","Type":"ContainerStarted","Data":"318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23"} Jul 13 22:02:04.720616 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:04.720581 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:02:04.721714 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:04.721684 2583 patch_prober.go:28] interesting pod/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 22:02:04.721839 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:04.721740 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 22:02:05.720002 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:05.719961 2583 patch_prober.go:28] interesting pod/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 22:02:05.720379 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:05.720027 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 22:02:06.719913 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:06.719867 2583 patch_prober.go:28] interesting pod/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 22:02:06.720114 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:06.719943 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 22:02:07.720681 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:07.720642 2583 patch_prober.go:28] interesting pod/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 13 22:02:07.721121 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:07.720717 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 13 22:02:08.723999 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:08.723969 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:02:08.724420 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:08.724352 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:02:08.725090 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:02:08.725071 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:03:07.255632 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:07.255594 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t"] Jul 13 22:03:07.256038 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:07.255820 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" containerID="cri-o://318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23" gracePeriod=30 Jul 13 22:03:08.726493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:08.726454 2583 patch_prober.go:28] interesting pod/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:03:08.726901 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:08.726512 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:03:12.508397 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.508369 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:03:12.542822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.542725 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-data\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.542822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.542808 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-credential-socket\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543024 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.542849 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsr69\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-kube-api-access-lsr69\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543024 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.542886 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-socket\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543024 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.542929 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-envoy\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543024 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.542983 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e767d066-973a-484c-9385-c20c7d9c7d2d-istiod-ca-cert\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543024 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543019 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-token\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543290 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543050 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-podinfo\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543290 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543056 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:03:12.543290 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543094 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-certs\") pod \"e767d066-973a-484c-9385-c20c7d9c7d2d\" (UID: \"e767d066-973a-484c-9385-c20c7d9c7d2d\") " Jul 13 22:03:12.543290 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543218 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:03:12.543504 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543399 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.543504 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543416 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.543622 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543573 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:03:12.543622 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543584 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e767d066-973a-484c-9385-c20c7d9c7d2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:03:12.543886 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.543864 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:03:12.545528 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.545497 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-kube-api-access-lsr69" (OuterVolumeSpecName: "kube-api-access-lsr69") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "kube-api-access-lsr69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:03:12.545716 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.545621 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:03:12.545908 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.545877 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:03:12.546051 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.546032 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e767d066-973a-484c-9385-c20c7d9c7d2d" (UID: "e767d066-973a-484c-9385-c20c7d9c7d2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:03:12.644813 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644789 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e767d066-973a-484c-9385-c20c7d9c7d2d-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.644813 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644811 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.644970 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644820 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.644970 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644828 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.644970 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644837 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.644970 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644846 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsr69\" (UniqueName: \"kubernetes.io/projected/e767d066-973a-484c-9385-c20c7d9c7d2d-kube-api-access-lsr69\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:12.644970 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:12.644854 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e767d066-973a-484c-9385-c20c7d9c7d2d-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:03:13.323327 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.323294 2583 generic.go:356] "Generic (PLEG): container finished" podID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerID="318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23" exitCode=0 Jul 13 22:03:13.323587 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.323344 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" Jul 13 22:03:13.323587 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.323344 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" event={"ID":"e767d066-973a-484c-9385-c20c7d9c7d2d","Type":"ContainerDied","Data":"318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23"} Jul 13 22:03:13.323587 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.323471 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t" event={"ID":"e767d066-973a-484c-9385-c20c7d9c7d2d","Type":"ContainerDied","Data":"79f2aec0fb84645d91604483317ff8133fe1773f106eb43d479d5c467266e2ba"} Jul 13 22:03:13.323587 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.323501 2583 scope.go:117] "RemoveContainer" containerID="318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23" Jul 13 22:03:13.334492 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.334472 2583 scope.go:117] "RemoveContainer" containerID="f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6" Jul 13 22:03:13.344238 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.344213 2583 scope.go:117] "RemoveContainer" containerID="318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23" Jul 13 22:03:13.344523 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:03:13.344499 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23\": container with ID starting with 318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23 not found: ID does not exist" containerID="318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23" Jul 13 22:03:13.344607 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.344530 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23"} err="failed to get container status \"318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23\": rpc error: code = NotFound desc = could not find container \"318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23\": container with ID starting with 318fc3d84f59db5a70e1c33fc589aecf842bd20d253f2379076ab3a2c45feb23 not found: ID does not exist" Jul 13 22:03:13.344607 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.344547 2583 scope.go:117] "RemoveContainer" containerID="f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6" Jul 13 22:03:13.344797 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:03:13.344763 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6\": container with ID starting with f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6 not found: ID does not exist" containerID="f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6" Jul 13 22:03:13.344843 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.344802 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6"} err="failed to get container status \"f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6\": rpc error: code = NotFound desc = could not find container \"f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6\": container with ID starting with f59d1e5bddd6b7cf9c13df7ba012f3a56932ef43f4066f99bae40ae0aa7b0da6 not found: ID does not exist" Jul 13 22:03:13.348704 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.348685 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t"] Jul 13 22:03:13.352915 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:13.352892 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-628bb6aa-openshift-default-5bc65c5995-wxd9t"] Jul 13 22:03:14.354652 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:03:14.354617 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" path="/var/lib/kubelet/pods/e767d066-973a-484c-9385-c20c7d9c7d2d/volumes" Jul 13 22:04:40.592882 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:40.592851 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:04:40.601163 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:40.601142 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:04:45.112871 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.112838 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2"] Jul 13 22:04:45.113367 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113347 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" Jul 13 22:04:45.113444 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113370 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" Jul 13 22:04:45.113444 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113403 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" Jul 13 22:04:45.113444 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113413 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" Jul 13 22:04:45.113444 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113434 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" Jul 13 22:04:45.113444 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113443 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" Jul 13 22:04:45.113694 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113457 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" Jul 13 22:04:45.113694 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113467 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" Jul 13 22:04:45.113694 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113552 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" Jul 13 22:04:45.113694 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113567 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" Jul 13 22:04:45.113694 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.113578 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a025a757-1b8f-4b0a-9387-44daeb63bd69" containerName="istio-proxy" Jul 13 22:04:45.116646 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.116626 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.119450 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.119426 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a941a48d-openshift-default-dockercfg-x5rsn\"" Jul 13 22:04:45.138616 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.138591 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2"] Jul 13 22:04:45.288874 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.288827 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58abb1aa-93ea-4c84-be54-a2307b9ca507-istiod-ca-cert\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289038 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.288879 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-podinfo\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289038 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.288941 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-envoy\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289038 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.288977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t59l9\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-kube-api-access-t59l9\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289038 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.289006 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-socket\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289239 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.289106 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-token\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289239 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.289174 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-data\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289239 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.289194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-credential-socket\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.289239 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.289213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-certs\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.389806 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.389717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-token\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.389806 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.389765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-data\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.389989 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.389963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-credential-socket\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390055 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-certs\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390055 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58abb1aa-93ea-4c84-be54-a2307b9ca507-istiod-ca-cert\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390192 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390090 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-data\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390192 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390094 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-podinfo\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390192 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-envoy\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390192 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t59l9\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-kube-api-access-t59l9\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390192 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-socket\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390465 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390396 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-certs\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390465 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390450 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-socket\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390616 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390598 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-credential-socket\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.390715 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.390691 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58abb1aa-93ea-4c84-be54-a2307b9ca507-istiod-ca-cert\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.392393 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.392373 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-envoy\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.392516 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.392499 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-podinfo\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.399510 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.399486 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-token\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.399917 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.399885 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t59l9\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-kube-api-access-t59l9\") pod \"e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.426852 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.426830 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:45.569322 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.569295 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2"] Jul 13 22:04:45.570547 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:04:45.570521 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58abb1aa_93ea_4c84_be54_a2307b9ca507.slice/crio-bbf62222107e2dea6d21fc758c2b23d73f816398b97a2262ec0bc3a864f72e7e WatchSource:0}: Error finding container bbf62222107e2dea6d21fc758c2b23d73f816398b97a2262ec0bc3a864f72e7e: Status 404 returned error can't find the container with id bbf62222107e2dea6d21fc758c2b23d73f816398b97a2262ec0bc3a864f72e7e Jul 13 22:04:45.572477 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.572445 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:04:45.572565 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.572521 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:04:45.572565 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.572549 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:04:45.815462 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.815424 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" event={"ID":"58abb1aa-93ea-4c84-be54-a2307b9ca507","Type":"ContainerStarted","Data":"c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9"} Jul 13 22:04:45.815632 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.815484 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" event={"ID":"58abb1aa-93ea-4c84-be54-a2307b9ca507","Type":"ContainerStarted","Data":"bbf62222107e2dea6d21fc758c2b23d73f816398b97a2262ec0bc3a864f72e7e"} Jul 13 22:04:45.843171 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:45.843124 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podStartSLOduration=0.843110078 podStartE2EDuration="843.110078ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:04:45.839985056 +0000 UTC m=+2106.659494864" watchObservedRunningTime="2026-07-13 22:04:45.843110078 +0000 UTC m=+2106.662619890" Jul 13 22:04:46.427184 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:46.427149 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:46.431661 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:46.431639 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:46.821339 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:46.821262 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:04:46.822195 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:04:46.822177 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:01.823161 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.823126 2583 patch_prober.go:28] interesting pod/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 22:05:01.823575 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.823191 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 22:05:01.907494 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.907463 2583 generic.go:356] "Generic (PLEG): container finished" podID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerID="c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9" exitCode=0 Jul 13 22:05:01.907669 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.907513 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" event={"ID":"58abb1aa-93ea-4c84-be54-a2307b9ca507","Type":"ContainerDied","Data":"c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9"} Jul 13 22:05:01.907939 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.907924 2583 scope.go:117] "RemoveContainer" containerID="c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9" Jul 13 22:05:01.908908 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.908877 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:05:01.908973 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.908945 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:05:01.909015 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:01.908993 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:05:02.915057 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:02.915022 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" event={"ID":"58abb1aa-93ea-4c84-be54-a2307b9ca507","Type":"ContainerStarted","Data":"b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a"} Jul 13 22:05:03.427511 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:03.427456 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:03.427869 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:03.427843 2583 patch_prober.go:28] interesting pod/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 22:05:03.427928 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:03.427897 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 22:05:04.427468 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:04.427420 2583 patch_prober.go:28] interesting pod/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 22:05:04.427878 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:04.427533 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 22:05:05.428249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:05.428209 2583 patch_prober.go:28] interesting pod/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 22:05:05.428633 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:05.428282 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 22:05:06.427467 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:06.427434 2583 patch_prober.go:28] interesting pod/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 13 22:05:06.427638 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:06.427518 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 13 22:05:07.431048 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:07.431016 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:07.431472 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:07.431310 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:07.431993 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:07.431978 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:25.997350 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:25.997312 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2"] Jul 13 22:05:25.997888 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:25.997545 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" containerID="cri-o://b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a" gracePeriod=30 Jul 13 22:05:31.245941 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.245917 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:31.342489 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342407 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-token\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342489 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342452 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-certs\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342489 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342485 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-data\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342531 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-podinfo\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342563 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-credential-socket\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342604 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-socket\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342640 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-envoy\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342687 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t59l9\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-kube-api-access-t59l9\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.342739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342719 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58abb1aa-93ea-4c84-be54-a2307b9ca507-istiod-ca-cert\") pod \"58abb1aa-93ea-4c84-be54-a2307b9ca507\" (UID: \"58abb1aa-93ea-4c84-be54-a2307b9ca507\") " Jul 13 22:05:31.343087 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342820 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:05:31.343087 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.342871 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:05:31.343208 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.343158 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:05:31.343208 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.343185 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58abb1aa-93ea-4c84-be54-a2307b9ca507-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:05:31.343327 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.343288 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.343327 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.343307 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.343440 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.343425 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-data" (OuterVolumeSpecName: "istio-data") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:05:31.344615 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.344592 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-token" (OuterVolumeSpecName: "istio-token") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:05:31.344989 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.344966 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:05:31.345078 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.345039 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:05:31.345287 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.345268 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-kube-api-access-t59l9" (OuterVolumeSpecName: "kube-api-access-t59l9") pod "58abb1aa-93ea-4c84-be54-a2307b9ca507" (UID: "58abb1aa-93ea-4c84-be54-a2307b9ca507"). InnerVolumeSpecName "kube-api-access-t59l9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444110 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444138 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444147 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444156 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444164 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58abb1aa-93ea-4c84-be54-a2307b9ca507-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444173 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t59l9\" (UniqueName: \"kubernetes.io/projected/58abb1aa-93ea-4c84-be54-a2307b9ca507-kube-api-access-t59l9\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:31.444181 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:31.444182 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58abb1aa-93ea-4c84-be54-a2307b9ca507-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:05:32.076822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.076760 2583 generic.go:356] "Generic (PLEG): container finished" podID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerID="b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a" exitCode=0 Jul 13 22:05:32.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.076831 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" Jul 13 22:05:32.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.076843 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" event={"ID":"58abb1aa-93ea-4c84-be54-a2307b9ca507","Type":"ContainerDied","Data":"b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a"} Jul 13 22:05:32.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.076899 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2" event={"ID":"58abb1aa-93ea-4c84-be54-a2307b9ca507","Type":"ContainerDied","Data":"bbf62222107e2dea6d21fc758c2b23d73f816398b97a2262ec0bc3a864f72e7e"} Jul 13 22:05:32.077000 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.076921 2583 scope.go:117] "RemoveContainer" containerID="b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a" Jul 13 22:05:32.086365 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.086347 2583 scope.go:117] "RemoveContainer" containerID="c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9" Jul 13 22:05:32.094359 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.094340 2583 scope.go:117] "RemoveContainer" containerID="b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a" Jul 13 22:05:32.094605 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:05:32.094585 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a\": container with ID starting with b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a not found: ID does not exist" containerID="b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a" Jul 13 22:05:32.094656 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.094612 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a"} err="failed to get container status \"b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a\": rpc error: code = NotFound desc = could not find container \"b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a\": container with ID starting with b76b595452053f5eedf29764442769f7ff351ca93d9b9a25fb8eb6ebd808337a not found: ID does not exist" Jul 13 22:05:32.094656 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.094628 2583 scope.go:117] "RemoveContainer" containerID="c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9" Jul 13 22:05:32.094876 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:05:32.094856 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9\": container with ID starting with c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9 not found: ID does not exist" containerID="c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9" Jul 13 22:05:32.094956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.094880 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9"} err="failed to get container status \"c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9\": rpc error: code = NotFound desc = could not find container \"c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9\": container with ID starting with c700f4c2e3876cb39270eac0f2224d039690efec73f52d2fe2115e5e5fb43dd9 not found: ID does not exist" Jul 13 22:05:32.099901 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.099880 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2"] Jul 13 22:05:32.104023 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.104003 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a941a48d-openshift-default-78dcc94cbc-2w9r2"] Jul 13 22:05:32.352254 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:32.352175 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" path="/var/lib/kubelet/pods/58abb1aa-93ea-4c84-be54-a2307b9ca507/volumes" Jul 13 22:05:46.132756 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.132720 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7"] Jul 13 22:05:46.133190 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.133111 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" Jul 13 22:05:46.133190 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.133124 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" Jul 13 22:05:46.133901 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.133871 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" Jul 13 22:05:46.134110 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.134092 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" Jul 13 22:05:46.134214 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.134197 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e767d066-973a-484c-9385-c20c7d9c7d2d" containerName="istio-proxy" Jul 13 22:05:46.141286 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.141259 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.144749 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.144721 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-06fcd095-openshift-default-dockercfg-ctx2d\"" Jul 13 22:05:46.158166 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.158141 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7"] Jul 13 22:05:46.180393 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istiod-ca-cert\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180527 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180411 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-token\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180527 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180471 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-credential-socket\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180700 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180591 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-certs\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180700 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-data\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180700 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180678 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-socket\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180878 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180711 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-podinfo\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180878 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180743 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-envoy\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.180878 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.180767 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk7d5\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-kube-api-access-kk7d5\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.281990 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.281952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istiod-ca-cert\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.281990 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-token\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-credential-socket\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-certs\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-data\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282219 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-socket\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-podinfo\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282548 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-envoy\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282548 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk7d5\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-kube-api-access-kk7d5\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282548 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282522 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-credential-socket\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282734 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282565 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-certs\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282734 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282674 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-socket\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282871 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282799 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istiod-ca-cert\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.282871 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.282837 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-data\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.284743 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.284721 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-envoy\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.285086 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.285065 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-podinfo\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.302293 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.302263 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk7d5\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-kube-api-access-kk7d5\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.302399 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.302266 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-token\") pod \"e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.452716 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.452685 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:46.608052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.608022 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7"] Jul 13 22:05:46.611685 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:05:46.611645 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1b6a81_aee3_4ca3_8b6a_b57ea61fb6f8.slice/crio-27ad6b7efe281b9211b382c160ff5a7e86f7cdea75146991abe125a2d3b40b2f WatchSource:0}: Error finding container 27ad6b7efe281b9211b382c160ff5a7e86f7cdea75146991abe125a2d3b40b2f: Status 404 returned error can't find the container with id 27ad6b7efe281b9211b382c160ff5a7e86f7cdea75146991abe125a2d3b40b2f Jul 13 22:05:46.614022 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.613986 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:05:46.614139 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.614066 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:05:46.614139 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:46.614098 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:05:47.160662 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:47.160628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" event={"ID":"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8","Type":"ContainerStarted","Data":"2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190"} Jul 13 22:05:47.161086 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:47.160690 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" event={"ID":"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8","Type":"ContainerStarted","Data":"27ad6b7efe281b9211b382c160ff5a7e86f7cdea75146991abe125a2d3b40b2f"} Jul 13 22:05:47.185110 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:47.185060 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podStartSLOduration=1.185043467 podStartE2EDuration="1.185043467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:05:47.182938114 +0000 UTC m=+2168.002447922" watchObservedRunningTime="2026-07-13 22:05:47.185043467 +0000 UTC m=+2168.004553276" Jul 13 22:05:47.453010 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:47.452969 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:47.458172 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:47.458150 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:48.166061 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:48.166023 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:05:48.167179 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:05:48.167161 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:06:02.242750 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:02.242715 2583 generic.go:356] "Generic (PLEG): container finished" podID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerID="2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190" exitCode=0 Jul 13 22:06:02.243175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:02.242812 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" event={"ID":"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8","Type":"ContainerDied","Data":"2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190"} Jul 13 22:06:02.243226 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:02.243197 2583 scope.go:117] "RemoveContainer" containerID="2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190" Jul 13 22:06:02.244205 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:02.244171 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:06:02.244313 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:02.244242 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:06:02.244313 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:02.244273 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:06:03.166335 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:03.166276 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:06:03.250019 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:03.249987 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" event={"ID":"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8","Type":"ContainerStarted","Data":"554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81"} Jul 13 22:06:03.453797 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:03.453749 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:06:03.454988 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:03.454965 2583 patch_prober.go:28] interesting pod/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 13 22:06:03.455062 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:03.455015 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 13 22:06:04.453435 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:04.453399 2583 patch_prober.go:28] interesting pod/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 13 22:06:04.453847 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:04.453453 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 13 22:06:05.454085 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:05.454037 2583 patch_prober.go:28] interesting pod/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 13 22:06:05.454481 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:05.454101 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 13 22:06:06.453453 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:06.453413 2583 patch_prober.go:28] interesting pod/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 13 22:06:06.453628 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:06.453482 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 13 22:06:07.457426 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:07.457396 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:06:07.457820 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:07.457797 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:06:07.458506 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:06:07.458488 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:07:49.080973 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:49.080939 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7"] Jul 13 22:07:49.081490 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:49.081151 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" containerID="cri-o://554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81" gracePeriod=30 Jul 13 22:07:52.460659 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:52.460617 2583 patch_prober.go:28] interesting pod/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:07:52.461049 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:52.460703 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:07:54.352484 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.352404 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:07:54.443208 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443168 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-credential-socket\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443406 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443226 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-certs\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443406 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443257 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-data\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443406 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443308 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-socket\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443406 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443350 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-token\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443406 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443377 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-envoy\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443678 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443406 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk7d5\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-kube-api-access-kk7d5\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443678 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443449 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istiod-ca-cert\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443678 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443519 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-podinfo\") pod \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\" (UID: \"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8\") " Jul 13 22:07:54.443678 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443599 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:07:54.443898 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.443843 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.444175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.444122 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:07:54.444303 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.444247 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:07:54.444303 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.444278 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:07:54.444435 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.444325 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:07:54.446145 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.446118 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:07:54.446268 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.446158 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:07:54.446341 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.446322 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-kube-api-access-kk7d5" (OuterVolumeSpecName: "kube-api-access-kk7d5") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "kube-api-access-kk7d5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:07:54.446397 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.446374 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" (UID: "bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:07:54.544557 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544521 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544557 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544551 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk7d5\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-kube-api-access-kk7d5\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544557 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544561 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544911 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544571 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544911 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544579 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544911 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544588 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544911 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544599 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.544911 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.544608 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:07:54.842728 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.842685 2583 generic.go:356] "Generic (PLEG): container finished" podID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerID="554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81" exitCode=0 Jul 13 22:07:54.842939 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.842739 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" Jul 13 22:07:54.842939 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.842766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" event={"ID":"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8","Type":"ContainerDied","Data":"554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81"} Jul 13 22:07:54.842939 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.842844 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7" event={"ID":"bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8","Type":"ContainerDied","Data":"27ad6b7efe281b9211b382c160ff5a7e86f7cdea75146991abe125a2d3b40b2f"} Jul 13 22:07:54.842939 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.842878 2583 scope.go:117] "RemoveContainer" containerID="554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81" Jul 13 22:07:54.852897 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.852852 2583 scope.go:117] "RemoveContainer" containerID="2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190" Jul 13 22:07:54.861914 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.861894 2583 scope.go:117] "RemoveContainer" containerID="554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81" Jul 13 22:07:54.862192 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:07:54.862174 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81\": container with ID starting with 554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81 not found: ID does not exist" containerID="554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81" Jul 13 22:07:54.862274 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.862204 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81"} err="failed to get container status \"554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81\": rpc error: code = NotFound desc = could not find container \"554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81\": container with ID starting with 554ead955836c4db9d8192679f347e5c454440c7ebcdcf8798dec28e66613c81 not found: ID does not exist" Jul 13 22:07:54.862274 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.862229 2583 scope.go:117] "RemoveContainer" containerID="2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190" Jul 13 22:07:54.862460 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:07:54.862443 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190\": container with ID starting with 2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190 not found: ID does not exist" containerID="2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190" Jul 13 22:07:54.862515 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.862467 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190"} err="failed to get container status \"2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190\": rpc error: code = NotFound desc = could not find container \"2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190\": container with ID starting with 2a0d88ac681efb2275f164af8ec1d343bad335d19b2b756bcb97bc5f600cb190 not found: ID does not exist" Jul 13 22:07:54.867394 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.867371 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7"] Jul 13 22:07:54.871289 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:54.871269 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-06fcd095-openshift-default-5bbd748699-nvxx7"] Jul 13 22:07:56.352526 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:07:56.352488 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" path="/var/lib/kubelet/pods/bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8/volumes" Jul 13 22:08:43.769756 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.769723 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x"] Jul 13 22:08:43.770164 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770144 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" Jul 13 22:08:43.770164 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770159 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" Jul 13 22:08:43.770243 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770175 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" Jul 13 22:08:43.770243 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770181 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" Jul 13 22:08:43.770243 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770195 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" Jul 13 22:08:43.770243 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770200 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58abb1aa-93ea-4c84-be54-a2307b9ca507" containerName="istio-proxy" Jul 13 22:08:43.770368 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.770267 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" Jul 13 22:08:43.774541 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.774516 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.777076 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.777055 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-160510-openshift-default-dockercfg-c9sjc\"" Jul 13 22:08:43.784341 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.784317 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x"] Jul 13 22:08:43.878464 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878429 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-certs\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878464 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878466 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-podinfo\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878682 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878487 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e28fc55-728f-4a27-b740-7a7fbb92d923-istiod-ca-cert\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878682 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878533 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59szm\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-kube-api-access-59szm\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878682 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878579 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-socket\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878682 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878597 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-credential-socket\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878682 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878619 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-data\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878905 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878707 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-envoy\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.878905 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.878729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-token\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979476 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979440 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-socket\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979660 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-credential-socket\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979660 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-data\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979660 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979613 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-envoy\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979660 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979646 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-token\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979943 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-certs\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979943 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979724 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-podinfo\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979943 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e28fc55-728f-4a27-b740-7a7fbb92d923-istiod-ca-cert\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.979943 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59szm\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-kube-api-access-59szm\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.980145 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.979975 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-credential-socket\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.980145 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.980107 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-certs\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.980145 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.980126 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-data\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.980332 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.980312 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-socket\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.980570 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.980552 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e28fc55-728f-4a27-b740-7a7fbb92d923-istiod-ca-cert\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.982535 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.982510 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-envoy\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.982641 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.982614 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-podinfo\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.988237 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.988207 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-token\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:43.988372 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:43.988358 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59szm\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-kube-api-access-59szm\") pod \"e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:44.088681 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:44.088606 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:44.230649 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:44.230620 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x"] Jul 13 22:08:44.232139 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:08:44.232115 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e28fc55_728f_4a27_b740_7a7fbb92d923.slice/crio-831de91d2b0815ff31faf0bec3db15e34da343323ed51e642225b9edbcee451b WatchSource:0}: Error finding container 831de91d2b0815ff31faf0bec3db15e34da343323ed51e642225b9edbcee451b: Status 404 returned error can't find the container with id 831de91d2b0815ff31faf0bec3db15e34da343323ed51e642225b9edbcee451b Jul 13 22:08:44.234344 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:44.234312 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:08:44.234439 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:44.234389 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:08:44.234500 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:44.234458 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:08:45.116095 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:45.116063 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" event={"ID":"2e28fc55-728f-4a27-b740-7a7fbb92d923","Type":"ContainerStarted","Data":"ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4"} Jul 13 22:08:45.116579 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:45.116124 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" event={"ID":"2e28fc55-728f-4a27-b740-7a7fbb92d923","Type":"ContainerStarted","Data":"831de91d2b0815ff31faf0bec3db15e34da343323ed51e642225b9edbcee451b"} Jul 13 22:08:45.148631 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:45.148579 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podStartSLOduration=2.148565689 podStartE2EDuration="2.148565689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:45.145531163 +0000 UTC m=+2345.965040971" watchObservedRunningTime="2026-07-13 22:08:45.148565689 +0000 UTC m=+2345.968075498" Jul 13 22:08:46.089530 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:46.089493 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:46.094370 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:46.094346 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:46.122556 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:46.122523 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:46.123511 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:46.123474 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:08:52.220635 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.220603 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg"] Jul 13 22:08:52.221133 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.221065 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1b6a81-aee3-4ca3-8b6a-b57ea61fb6f8" containerName="istio-proxy" Jul 13 22:08:52.224173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.224153 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.226703 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.226672 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3394b0-openshift-default-dockercfg-f8blp\"" Jul 13 22:08:52.236217 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.236196 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg"] Jul 13 22:08:52.353293 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.353264 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ada08c-2329-4d5c-beb6-92ace8b3e369-istiod-ca-cert\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.353463 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.353304 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-token\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.353577 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.353393 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-credential-socket\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.359512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.353858 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thztw\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-kube-api-access-thztw\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.359512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.353979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-data\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.359512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.354022 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-envoy\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.359512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.354071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-podinfo\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.359512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.354108 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-certs\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.359512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.354151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-socket\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.454702 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.454662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-data\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.454702 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.454706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-envoy\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.454934 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.454728 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-podinfo\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.454934 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.454750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-certs\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455015 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.454932 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-socket\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455015 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.454990 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ada08c-2329-4d5c-beb6-92ace8b3e369-istiod-ca-cert\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455105 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455031 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-token\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455156 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-credential-socket\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455156 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455133 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-certs\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455156 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455144 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thztw\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-kube-api-access-thztw\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455325 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455209 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-data\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455448 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455423 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-socket\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455526 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455468 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-credential-socket\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.455736 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.455714 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ada08c-2329-4d5c-beb6-92ace8b3e369-istiod-ca-cert\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.457585 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.457559 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-envoy\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.457906 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.457885 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-podinfo\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.463295 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.463276 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-token\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.463447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.463427 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thztw\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-kube-api-access-thztw\") pod \"e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.535053 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.534950 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:52.673590 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.673564 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg"] Jul 13 22:08:52.675848 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:08:52.675817 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ada08c_2329_4d5c_beb6_92ace8b3e369.slice/crio-2e14b898725c91acf8d1564a5b5341a5182322372d9a634e0f0fcd6ad168183d WatchSource:0}: Error finding container 2e14b898725c91acf8d1564a5b5341a5182322372d9a634e0f0fcd6ad168183d: Status 404 returned error can't find the container with id 2e14b898725c91acf8d1564a5b5341a5182322372d9a634e0f0fcd6ad168183d Jul 13 22:08:52.677992 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.677951 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:08:52.678100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.678057 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:08:52.678146 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:52.678112 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:08:53.161787 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:53.161738 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" event={"ID":"85ada08c-2329-4d5c-beb6-92ace8b3e369","Type":"ContainerStarted","Data":"67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96"} Jul 13 22:08:53.161954 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:53.161823 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" event={"ID":"85ada08c-2329-4d5c-beb6-92ace8b3e369","Type":"ContainerStarted","Data":"2e14b898725c91acf8d1564a5b5341a5182322372d9a634e0f0fcd6ad168183d"} Jul 13 22:08:53.186471 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:53.186428 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podStartSLOduration=1.186409572 podStartE2EDuration="1.186409572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:53.184722804 +0000 UTC m=+2354.004232664" watchObservedRunningTime="2026-07-13 22:08:53.186409572 +0000 UTC m=+2354.005919380" Jul 13 22:08:53.535602 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:53.535564 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:53.540568 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:53.540543 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:54.167202 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:54.167172 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:08:54.168323 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:08:54.168303 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:09:02.163992 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:02.163950 2583 patch_prober.go:28] interesting pod/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 22:09:02.164368 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:02.164027 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 22:09:03.230317 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:03.230282 2583 generic.go:356] "Generic (PLEG): container finished" podID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerID="ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4" exitCode=0 Jul 13 22:09:03.230726 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:03.230355 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" event={"ID":"2e28fc55-728f-4a27-b740-7a7fbb92d923","Type":"ContainerDied","Data":"ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4"} Jul 13 22:09:03.230787 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:03.230742 2583 scope.go:117] "RemoveContainer" containerID="ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4" Jul 13 22:09:03.231872 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:03.231837 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:09:03.231948 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:03.231903 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:09:03.231948 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:03.231937 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:09:04.237931 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:04.237899 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" event={"ID":"2e28fc55-728f-4a27-b740-7a7fbb92d923","Type":"ContainerStarted","Data":"042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d"} Jul 13 22:09:05.089597 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:05.089558 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:09:05.089941 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:05.089609 2583 patch_prober.go:28] interesting pod/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 22:09:05.090134 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:05.090106 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 22:09:06.089924 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:06.089882 2583 patch_prober.go:28] interesting pod/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 22:09:06.090444 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:06.089948 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 22:09:07.089874 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:07.089414 2583 patch_prober.go:28] interesting pod/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 22:09:07.089874 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:07.089475 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 22:09:08.089670 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:08.089627 2583 patch_prober.go:28] interesting pod/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 22:09:08.089902 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:08.089740 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 22:09:09.092428 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.092401 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:09:09.094842 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.092869 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:09:09.094842 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.093507 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:09:09.168852 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.168826 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 22:09:09.168972 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.168877 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 22:09:09.266841 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.266809 2583 generic.go:356] "Generic (PLEG): container finished" podID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerID="67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96" exitCode=0 Jul 13 22:09:09.266986 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.266876 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" event={"ID":"85ada08c-2329-4d5c-beb6-92ace8b3e369","Type":"ContainerDied","Data":"67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96"} Jul 13 22:09:09.267283 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.267270 2583 scope.go:117] "RemoveContainer" containerID="67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96" Jul 13 22:09:09.268264 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.268237 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:09:09.268338 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.268320 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:09:09.268387 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:09.268372 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:09:10.301089 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:10.301055 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" event={"ID":"85ada08c-2329-4d5c-beb6-92ace8b3e369","Type":"ContainerStarted","Data":"7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6"} Jul 13 22:09:10.535724 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:10.535684 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:09:10.535929 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:10.535896 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 22:09:10.535974 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:10.535946 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 22:09:11.535319 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:11.535277 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 22:09:11.535809 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:11.535340 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 22:09:12.536304 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:12.536266 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 22:09:12.536711 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:12.536323 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 22:09:13.536122 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:13.536086 2583 patch_prober.go:28] interesting pod/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 13 22:09:13.536293 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:13.536140 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 13 22:09:14.539596 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:14.539566 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:09:14.539989 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:14.539945 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:09:14.540754 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:14.540733 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:09:40.630349 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:40.630317 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:09:40.642578 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:09:40.642553 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:10:19.831108 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:19.831076 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x"] Jul 13 22:10:19.831648 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:19.831297 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" containerID="cri-o://042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d" gracePeriod=30 Jul 13 22:10:24.095069 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:24.095023 2583 patch_prober.go:28] interesting pod/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:10:24.095448 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:24.095096 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:10:25.092864 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.092803 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:10:25.172988 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.172959 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-socket\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.172988 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.172994 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-podinfo\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173037 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-envoy\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173069 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-data\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173088 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e28fc55-728f-4a27-b740-7a7fbb92d923-istiod-ca-cert\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173116 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59szm\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-kube-api-access-59szm\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173161 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-credential-socket\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173289 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-token\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173287 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:25.173493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173335 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-certs\") pod \"2e28fc55-728f-4a27-b740-7a7fbb92d923\" (UID: \"2e28fc55-728f-4a27-b740-7a7fbb92d923\") " Jul 13 22:10:25.173956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173589 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:25.173956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173627 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:25.173956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173739 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.173956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173760 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.173956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173791 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.173956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.173861 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e28fc55-728f-4a27-b740-7a7fbb92d923-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:10:25.174291 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.174066 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:25.175411 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.175379 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:10:25.175560 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.175403 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-kube-api-access-59szm" (OuterVolumeSpecName: "kube-api-access-59szm") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "kube-api-access-59szm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:10:25.175560 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.175440 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:10:25.175560 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.175448 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e28fc55-728f-4a27-b740-7a7fbb92d923" (UID: "2e28fc55-728f-4a27-b740-7a7fbb92d923"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:25.274465 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.274440 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.274465 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.274461 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.274613 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.274482 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.274613 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.274491 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e28fc55-728f-4a27-b740-7a7fbb92d923-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.274613 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.274499 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59szm\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-kube-api-access-59szm\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.274613 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.274507 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e28fc55-728f-4a27-b740-7a7fbb92d923-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:25.705989 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.705956 2583 generic.go:356] "Generic (PLEG): container finished" podID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerID="042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d" exitCode=0 Jul 13 22:10:25.706140 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.706008 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" Jul 13 22:10:25.706140 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.706043 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" event={"ID":"2e28fc55-728f-4a27-b740-7a7fbb92d923","Type":"ContainerDied","Data":"042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d"} Jul 13 22:10:25.706140 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.706099 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x" event={"ID":"2e28fc55-728f-4a27-b740-7a7fbb92d923","Type":"ContainerDied","Data":"831de91d2b0815ff31faf0bec3db15e34da343323ed51e642225b9edbcee451b"} Jul 13 22:10:25.706140 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.706122 2583 scope.go:117] "RemoveContainer" containerID="042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d" Jul 13 22:10:25.715801 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.715768 2583 scope.go:117] "RemoveContainer" containerID="ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4" Jul 13 22:10:25.724421 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.724406 2583 scope.go:117] "RemoveContainer" containerID="042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d" Jul 13 22:10:25.724661 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:10:25.724645 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d\": container with ID starting with 042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d not found: ID does not exist" containerID="042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d" Jul 13 22:10:25.724707 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.724669 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d"} err="failed to get container status \"042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d\": rpc error: code = NotFound desc = could not find container \"042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d\": container with ID starting with 042ae99972256dbe5583fa748a5697a3450a1283130ddc17584e6f6d2bcf670d not found: ID does not exist" Jul 13 22:10:25.724707 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.724685 2583 scope.go:117] "RemoveContainer" containerID="ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4" Jul 13 22:10:25.724944 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:10:25.724927 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4\": container with ID starting with ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4 not found: ID does not exist" containerID="ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4" Jul 13 22:10:25.724995 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.724947 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4"} err="failed to get container status \"ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4\": rpc error: code = NotFound desc = could not find container \"ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4\": container with ID starting with ec7190172baa4c4149e277c10d6bdded08b5f3c8b6510364f57ab796e02e71c4 not found: ID does not exist" Jul 13 22:10:25.729435 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.729414 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x"] Jul 13 22:10:25.733846 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:25.733826 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-160510-openshift-default-6ffcb9f6c-b5h7x"] Jul 13 22:10:26.352529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:26.352498 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" path="/var/lib/kubelet/pods/2e28fc55-728f-4a27-b740-7a7fbb92d923/volumes" Jul 13 22:10:45.958739 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:45.958702 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg"] Jul 13 22:10:45.959224 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:45.958909 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" containerID="cri-o://7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6" gracePeriod=30 Jul 13 22:10:46.849342 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.849310 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g"] Jul 13 22:10:46.849698 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.849686 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" Jul 13 22:10:46.849743 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.849699 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" Jul 13 22:10:46.849796 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.849765 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" Jul 13 22:10:46.849796 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.849794 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" Jul 13 22:10:46.852736 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.852715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.855292 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.855265 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-785879-openshift-default-dockercfg-sjqpx\"" Jul 13 22:10:46.869877 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.869857 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g"] Jul 13 22:10:46.950155 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-token\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950316 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950172 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-credential-socket\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950316 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950205 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-socket\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950316 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-certs\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950316 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950252 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-podinfo\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950316 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-envoy\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950581 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23881415-ce76-42c0-a9f3-638f6bf785d6-istiod-ca-cert\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950581 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950400 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-data\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:46.950581 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:46.950466 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxsl\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-kube-api-access-rfxsl\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.050930 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.050898 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-socket\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.050930 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.050933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-certs\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.050959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-podinfo\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-envoy\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23881415-ce76-42c0-a9f3-638f6bf785d6-istiod-ca-cert\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051328 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-data\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxsl\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-kube-api-access-rfxsl\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051423 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-certs\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-token\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051437 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051374 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-socket\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051868 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051497 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-credential-socket\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051868 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051629 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-data\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.051868 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.051805 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-credential-socket\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.052093 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.052069 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23881415-ce76-42c0-a9f3-638f6bf785d6-istiod-ca-cert\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.053466 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.053445 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-envoy\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.053615 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.053597 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-podinfo\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.060353 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.060326 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxsl\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-kube-api-access-rfxsl\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.060658 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.060641 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-token\") pod \"e2e-api-a-785879-openshift-default-f4755947-vlp8g\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.162609 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.162526 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:47.301873 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.301846 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g"] Jul 13 22:10:47.303521 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:10:47.303495 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23881415_ce76_42c0_a9f3_638f6bf785d6.slice/crio-d03a930fa6b7615fdc5fef1a52a36c8dd9f177b21154195934102434721ac58e WatchSource:0}: Error finding container d03a930fa6b7615fdc5fef1a52a36c8dd9f177b21154195934102434721ac58e: Status 404 returned error can't find the container with id d03a930fa6b7615fdc5fef1a52a36c8dd9f177b21154195934102434721ac58e Jul 13 22:10:47.305597 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.305568 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:10:47.305659 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.305631 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:10:47.305699 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.305660 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:10:47.833206 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.833165 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" event={"ID":"23881415-ce76-42c0-a9f3-638f6bf785d6","Type":"ContainerStarted","Data":"08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64"} Jul 13 22:10:47.833375 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.833265 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" event={"ID":"23881415-ce76-42c0-a9f3-638f6bf785d6","Type":"ContainerStarted","Data":"d03a930fa6b7615fdc5fef1a52a36c8dd9f177b21154195934102434721ac58e"} Jul 13 22:10:47.854491 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:47.854438 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podStartSLOduration=1.8544194040000002 podStartE2EDuration="1.854419404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:10:47.852954628 +0000 UTC m=+2468.672464471" watchObservedRunningTime="2026-07-13 22:10:47.854419404 +0000 UTC m=+2468.673929215" Jul 13 22:10:48.163058 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:48.162986 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:48.167741 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:48.167718 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:48.839386 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:48.839355 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:48.840411 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:48.840390 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:10:51.207298 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.207276 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:10:51.390827 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390740 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-envoy\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.390827 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390788 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-certs\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.390827 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390824 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-token\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390855 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-socket\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390872 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-podinfo\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390893 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-data\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390917 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ada08c-2329-4d5c-beb6-92ace8b3e369-istiod-ca-cert\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390966 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-credential-socket\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391100 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.390991 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thztw\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-kube-api-access-thztw\") pod \"85ada08c-2329-4d5c-beb6-92ace8b3e369\" (UID: \"85ada08c-2329-4d5c-beb6-92ace8b3e369\") " Jul 13 22:10:51.391414 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.391130 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:51.391414 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.391278 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.391525 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.391439 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:51.391525 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.391501 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:51.391637 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.391566 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ada08c-2329-4d5c-beb6-92ace8b3e369-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:10:51.391763 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.391729 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-data" (OuterVolumeSpecName: "istio-data") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:51.393116 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.393088 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:10:51.393116 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.393100 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:10:51.393253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.393202 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-kube-api-access-thztw" (OuterVolumeSpecName: "kube-api-access-thztw") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "kube-api-access-thztw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:10:51.393253 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.393209 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-token" (OuterVolumeSpecName: "istio-token") pod "85ada08c-2329-4d5c-beb6-92ace8b3e369" (UID: "85ada08c-2329-4d5c-beb6-92ace8b3e369"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:10:51.492114 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492092 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492114 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492114 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492260 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492124 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492260 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492133 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492260 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492143 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492260 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492151 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ada08c-2329-4d5c-beb6-92ace8b3e369-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492260 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492159 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ada08c-2329-4d5c-beb6-92ace8b3e369-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.492260 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.492168 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thztw\" (UniqueName: \"kubernetes.io/projected/85ada08c-2329-4d5c-beb6-92ace8b3e369-kube-api-access-thztw\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:10:51.856433 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.856404 2583 generic.go:356] "Generic (PLEG): container finished" podID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerID="7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6" exitCode=0 Jul 13 22:10:51.856618 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.856447 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" Jul 13 22:10:51.856618 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.856466 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" event={"ID":"85ada08c-2329-4d5c-beb6-92ace8b3e369","Type":"ContainerDied","Data":"7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6"} Jul 13 22:10:51.856618 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.856516 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg" event={"ID":"85ada08c-2329-4d5c-beb6-92ace8b3e369","Type":"ContainerDied","Data":"2e14b898725c91acf8d1564a5b5341a5182322372d9a634e0f0fcd6ad168183d"} Jul 13 22:10:51.856618 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.856549 2583 scope.go:117] "RemoveContainer" containerID="7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6" Jul 13 22:10:51.866960 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.866943 2583 scope.go:117] "RemoveContainer" containerID="67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96" Jul 13 22:10:51.875363 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.875345 2583 scope.go:117] "RemoveContainer" containerID="7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6" Jul 13 22:10:51.875653 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:10:51.875634 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6\": container with ID starting with 7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6 not found: ID does not exist" containerID="7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6" Jul 13 22:10:51.875717 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.875667 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6"} err="failed to get container status \"7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6\": rpc error: code = NotFound desc = could not find container \"7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6\": container with ID starting with 7f7cb3219e21cae78fce62fdd0bb7e4063a2515cb6acef40498050b0e02092d6 not found: ID does not exist" Jul 13 22:10:51.875717 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.875685 2583 scope.go:117] "RemoveContainer" containerID="67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96" Jul 13 22:10:51.875998 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:10:51.875973 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96\": container with ID starting with 67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96 not found: ID does not exist" containerID="67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96" Jul 13 22:10:51.876058 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.876005 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96"} err="failed to get container status \"67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96\": rpc error: code = NotFound desc = could not find container \"67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96\": container with ID starting with 67609ee1318ee99d825f41eedb1f7cade6da7c5d4bb501de05c83edf0ab85e96 not found: ID does not exist" Jul 13 22:10:51.879686 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.879666 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg"] Jul 13 22:10:51.883367 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:51.883343 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3394b0-openshift-default-dd4fcdd45-d7xjg"] Jul 13 22:10:52.351837 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:52.351805 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" path="/var/lib/kubelet/pods/85ada08c-2329-4d5c-beb6-92ace8b3e369/volumes" Jul 13 22:10:55.073145 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073110 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n"] Jul 13 22:10:55.073529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073485 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" Jul 13 22:10:55.073529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073495 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" Jul 13 22:10:55.073529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073502 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" Jul 13 22:10:55.073529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073508 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e28fc55-728f-4a27-b740-7a7fbb92d923" containerName="istio-proxy" Jul 13 22:10:55.073529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073522 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" Jul 13 22:10:55.073529 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073528 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" Jul 13 22:10:55.073745 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073597 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" Jul 13 22:10:55.073745 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.073605 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ada08c-2329-4d5c-beb6-92ace8b3e369" containerName="istio-proxy" Jul 13 22:10:55.077947 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.077927 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.080403 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.080383 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9dff5a-openshift-default-dockercfg-n7t5h\"" Jul 13 22:10:55.088816 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.088791 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n"] Jul 13 22:10:55.224054 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-data\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224248 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224063 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-credential-socket\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224248 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224114 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-token\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224248 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224208 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9zzm\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-kube-api-access-d9zzm\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224430 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istiod-ca-cert\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224430 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224315 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-certs\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224430 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224352 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-socket\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224430 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224387 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-envoy\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.224585 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.224453 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-podinfo\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.325650 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-socket\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.325650 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-envoy\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.325650 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325626 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-podinfo\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325749 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-data\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-credential-socket\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-token\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325931 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9zzm\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-kube-api-access-d9zzm\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.325991 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istiod-ca-cert\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.326030 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-socket\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.326050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-certs\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.326089 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-data\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326175 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.326153 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-credential-socket\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326592 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.326391 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-certs\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.326696 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.326678 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istiod-ca-cert\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.328099 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.328071 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-envoy\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.328235 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.328218 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-podinfo\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.339330 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.339310 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-token\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.339505 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.339488 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9zzm\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-kube-api-access-d9zzm\") pod \"e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.389581 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.389548 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:55.528951 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.528913 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n"] Jul 13 22:10:55.530939 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:10:55.530911 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b6b01b_4a53_42db_88de_eb3dec95a0f8.slice/crio-d01db341777bc4e51bd7893e4fa1dc2aa711803ba03a558b91e8e684db6e3137 WatchSource:0}: Error finding container d01db341777bc4e51bd7893e4fa1dc2aa711803ba03a558b91e8e684db6e3137: Status 404 returned error can't find the container with id d01db341777bc4e51bd7893e4fa1dc2aa711803ba03a558b91e8e684db6e3137 Jul 13 22:10:55.533467 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.533434 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:10:55.533584 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.533503 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:10:55.533584 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.533534 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:10:55.883966 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.883860 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" event={"ID":"d2b6b01b-4a53-42db-88de-eb3dec95a0f8","Type":"ContainerStarted","Data":"78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d"} Jul 13 22:10:55.883966 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.883935 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" event={"ID":"d2b6b01b-4a53-42db-88de-eb3dec95a0f8","Type":"ContainerStarted","Data":"d01db341777bc4e51bd7893e4fa1dc2aa711803ba03a558b91e8e684db6e3137"} Jul 13 22:10:55.909998 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:55.909944 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podStartSLOduration=0.909926752 podStartE2EDuration="909.926752ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:10:55.906733104 +0000 UTC m=+2476.726242940" watchObservedRunningTime="2026-07-13 22:10:55.909926752 +0000 UTC m=+2476.729436561" Jul 13 22:10:56.390863 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:56.390822 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:56.400722 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:56.400691 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:56.893854 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:56.893819 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:10:56.894832 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:10:56.894802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:11:02.933314 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:02.933279 2583 generic.go:356] "Generic (PLEG): container finished" podID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerID="08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64" exitCode=0 Jul 13 22:11:02.933729 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:02.933335 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" event={"ID":"23881415-ce76-42c0-a9f3-638f6bf785d6","Type":"ContainerDied","Data":"08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64"} Jul 13 22:11:02.933729 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:02.933706 2583 scope.go:117] "RemoveContainer" containerID="08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64" Jul 13 22:11:02.934765 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:02.934732 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:11:02.934898 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:02.934855 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:11:02.934965 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:02.934908 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:11:03.839653 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:03.839617 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:03.941109 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:03.941070 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" event={"ID":"23881415-ce76-42c0-a9f3-638f6bf785d6","Type":"ContainerStarted","Data":"b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652"} Jul 13 22:11:04.163394 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:04.163289 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:04.164504 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:04.164481 2583 patch_prober.go:28] interesting pod/e2e-api-a-785879-openshift-default-f4755947-vlp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 13 22:11:04.164569 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:04.164528 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 13 22:11:05.163672 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:05.163632 2583 patch_prober.go:28] interesting pod/e2e-api-a-785879-openshift-default-f4755947-vlp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 13 22:11:05.164081 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:05.163708 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 13 22:11:06.163940 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:06.163902 2583 patch_prober.go:28] interesting pod/e2e-api-a-785879-openshift-default-f4755947-vlp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 13 22:11:06.164331 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:06.163960 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 13 22:11:07.163057 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:07.163016 2583 patch_prober.go:28] interesting pod/e2e-api-a-785879-openshift-default-f4755947-vlp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 13 22:11:07.163251 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:07.163077 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 13 22:11:08.167202 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:08.167172 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:08.167658 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:08.167458 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:08.168239 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:08.168219 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:12.935956 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:12.935914 2583 patch_prober.go:28] interesting pod/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 22:11:12.936376 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:12.935986 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 22:11:15.013590 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:15.013556 2583 generic.go:356] "Generic (PLEG): container finished" podID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerID="78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d" exitCode=0 Jul 13 22:11:15.014066 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:15.013628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" event={"ID":"d2b6b01b-4a53-42db-88de-eb3dec95a0f8","Type":"ContainerDied","Data":"78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d"} Jul 13 22:11:15.014114 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:15.014068 2583 scope.go:117] "RemoveContainer" containerID="78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d" Jul 13 22:11:15.015101 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:15.015072 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:11:15.015173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:15.015138 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:11:15.015210 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:15.015171 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:11:16.021886 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:16.021851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" event={"ID":"d2b6b01b-4a53-42db-88de-eb3dec95a0f8","Type":"ContainerStarted","Data":"40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312"} Jul 13 22:11:16.390247 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:16.390132 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:11:16.390467 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:16.390437 2583 patch_prober.go:28] interesting pod/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 13 22:11:16.390551 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:16.390519 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 13 22:11:17.391150 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:17.391101 2583 patch_prober.go:28] interesting pod/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 13 22:11:17.391636 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:17.391188 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 13 22:11:18.391059 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:18.390584 2583 patch_prober.go:28] interesting pod/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 13 22:11:18.391059 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:18.390649 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 13 22:11:19.390886 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:19.390847 2583 patch_prober.go:28] interesting pod/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 13 22:11:19.391601 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:19.391195 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 13 22:11:20.394004 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:20.393975 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:11:20.394634 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:20.394247 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:11:20.394988 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:20.394971 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:11:36.143715 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:36.143632 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g"] Jul 13 22:11:36.144230 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:36.143911 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" containerID="cri-o://b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652" gracePeriod=30 Jul 13 22:11:38.170590 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:38.170554 2583 patch_prober.go:28] interesting pod/e2e-api-a-785879-openshift-default-f4755947-vlp8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:11:38.171006 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:38.170622 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:11:41.395807 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.395782 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:41.469930 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.469900 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-data\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.469930 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.469935 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-credential-socket\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470147 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.469991 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-certs\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470147 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470055 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfxsl\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-kube-api-access-rfxsl\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470147 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470088 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-podinfo\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470147 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470105 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23881415-ce76-42c0-a9f3-638f6bf785d6-istiod-ca-cert\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470360 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470185 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-token\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470360 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470209 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:11:41.470360 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470223 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-envoy\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470360 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470280 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-socket\") pod \"23881415-ce76-42c0-a9f3-638f6bf785d6\" (UID: \"23881415-ce76-42c0-a9f3-638f6bf785d6\") " Jul 13 22:11:41.470573 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470219 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:11:41.470630 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470586 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.470630 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470601 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:11:41.470630 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470608 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.470821 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470765 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23881415-ce76-42c0-a9f3-638f6bf785d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:11:41.470888 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.470816 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:11:41.472308 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.472281 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-kube-api-access-rfxsl" (OuterVolumeSpecName: "kube-api-access-rfxsl") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "kube-api-access-rfxsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:11:41.472438 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.472328 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:11:41.472605 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.472584 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:11:41.472691 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.472629 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23881415-ce76-42c0-a9f3-638f6bf785d6" (UID: "23881415-ce76-42c0-a9f3-638f6bf785d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:11:41.571937 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571908 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfxsl\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-kube-api-access-rfxsl\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.571937 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571934 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.571937 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571945 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23881415-ce76-42c0-a9f3-638f6bf785d6-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.572158 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571954 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.572158 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571961 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.572158 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571969 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:41.572158 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:41.571976 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23881415-ce76-42c0-a9f3-638f6bf785d6-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:11:42.174597 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.174554 2583 generic.go:356] "Generic (PLEG): container finished" podID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerID="b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652" exitCode=0 Jul 13 22:11:42.174792 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.174605 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" Jul 13 22:11:42.174792 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.174640 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" event={"ID":"23881415-ce76-42c0-a9f3-638f6bf785d6","Type":"ContainerDied","Data":"b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652"} Jul 13 22:11:42.174792 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.174695 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g" event={"ID":"23881415-ce76-42c0-a9f3-638f6bf785d6","Type":"ContainerDied","Data":"d03a930fa6b7615fdc5fef1a52a36c8dd9f177b21154195934102434721ac58e"} Jul 13 22:11:42.174792 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.174719 2583 scope.go:117] "RemoveContainer" containerID="b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652" Jul 13 22:11:42.183901 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.183883 2583 scope.go:117] "RemoveContainer" containerID="08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64" Jul 13 22:11:42.193893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.193867 2583 scope.go:117] "RemoveContainer" containerID="b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652" Jul 13 22:11:42.194238 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:11:42.194218 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652\": container with ID starting with b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652 not found: ID does not exist" containerID="b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652" Jul 13 22:11:42.194306 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.194246 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652"} err="failed to get container status \"b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652\": rpc error: code = NotFound desc = could not find container \"b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652\": container with ID starting with b456b21d640c208ae9be9e5d7dc7299783fa33d3722997e665cceb5cd537d652 not found: ID does not exist" Jul 13 22:11:42.194306 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.194262 2583 scope.go:117] "RemoveContainer" containerID="08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64" Jul 13 22:11:42.194525 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:11:42.194507 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64\": container with ID starting with 08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64 not found: ID does not exist" containerID="08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64" Jul 13 22:11:42.194585 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.194532 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64"} err="failed to get container status \"08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64\": rpc error: code = NotFound desc = could not find container \"08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64\": container with ID starting with 08c31171a281587536b22770cb2a8aa1879ea759d98ef59fe5da59b151c4ec64 not found: ID does not exist" Jul 13 22:11:42.198851 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.198830 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g"] Jul 13 22:11:42.204931 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.204910 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-785879-openshift-default-f4755947-vlp8g"] Jul 13 22:11:42.351685 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:11:42.351650 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" path="/var/lib/kubelet/pods/23881415-ce76-42c0-a9f3-638f6bf785d6/volumes" Jul 13 22:12:02.246644 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:02.246609 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n"] Jul 13 22:12:02.247090 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:02.246874 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" containerID="cri-o://40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312" gracePeriod=30 Jul 13 22:12:03.123728 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.123699 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd"] Jul 13 22:12:03.124120 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.124107 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" Jul 13 22:12:03.124172 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.124122 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" Jul 13 22:12:03.124210 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.124200 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" Jul 13 22:12:03.127207 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.127186 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.129701 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.129684 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f89343-openshift-default-dockercfg-f22mp\"" Jul 13 22:12:03.139723 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.139699 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd"] Jul 13 22:12:03.261710 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.261682 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-istio-token\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.261749 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-credential-socket\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.261860 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flpq7\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-kube-api-access-flpq7\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.261908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/361c18b5-3e40-4650-9f2a-94904b96609f-istiod-ca-cert\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.261962 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-envoy\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.261998 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-certs\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.262015 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/361c18b5-3e40-4650-9f2a-94904b96609f-istio-podinfo\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.262038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-data\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.262151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.262111 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-socket\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362690 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-istio-token\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-credential-socket\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flpq7\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-kube-api-access-flpq7\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/361c18b5-3e40-4650-9f2a-94904b96609f-istiod-ca-cert\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-envoy\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362868 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-certs\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.362893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362889 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/361c18b5-3e40-4650-9f2a-94904b96609f-istio-podinfo\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363216 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-data\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363216 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.362984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-socket\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363216 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.363192 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-credential-socket\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363359 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.363292 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-certs\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363413 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.363383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-socket\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363474 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.363462 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-data\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.363577 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.363554 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/361c18b5-3e40-4650-9f2a-94904b96609f-istiod-ca-cert\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.365007 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.364985 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-envoy\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.365381 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.365361 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/361c18b5-3e40-4650-9f2a-94904b96609f-istio-podinfo\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.372552 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.372532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-istio-token\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.372857 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.372836 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flpq7\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-kube-api-access-flpq7\") pod \"e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.437402 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.437374 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:03.578215 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.578182 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd"] Jul 13 22:12:03.582455 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:12:03.582423 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361c18b5_3e40_4650_9f2a_94904b96609f.slice/crio-99845e5e3e4d678721708639a36fb1ebdffad8417336c87579097320fd0b945e WatchSource:0}: Error finding container 99845e5e3e4d678721708639a36fb1ebdffad8417336c87579097320fd0b945e: Status 404 returned error can't find the container with id 99845e5e3e4d678721708639a36fb1ebdffad8417336c87579097320fd0b945e Jul 13 22:12:03.585259 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.585219 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:03.585342 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.585300 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:03.585411 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:03.585336 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:04.292575 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:04.292541 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" event={"ID":"361c18b5-3e40-4650-9f2a-94904b96609f","Type":"ContainerStarted","Data":"a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7"} Jul 13 22:12:04.292963 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:04.292595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" event={"ID":"361c18b5-3e40-4650-9f2a-94904b96609f","Type":"ContainerStarted","Data":"99845e5e3e4d678721708639a36fb1ebdffad8417336c87579097320fd0b945e"} Jul 13 22:12:04.315301 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:04.315254 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" podStartSLOduration=1.315238377 podStartE2EDuration="1.315238377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:12:04.312871813 +0000 UTC m=+2545.132381626" watchObservedRunningTime="2026-07-13 22:12:04.315238377 +0000 UTC m=+2545.134748185" Jul 13 22:12:04.438026 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:04.437990 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:04.442562 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:04.442536 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:05.297931 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:05.297900 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:05.298883 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:05.298860 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:05.397118 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:05.397080 2583 patch_prober.go:28] interesting pod/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:12:05.397249 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:05.397138 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:12:07.504915 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.504892 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:12:07.604573 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604496 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-envoy\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.604573 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604537 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-socket\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.604737 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604583 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-token\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.604737 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604638 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-podinfo\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.604867 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604753 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9zzm\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-kube-api-access-d9zzm\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.604867 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604817 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-data\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.604867 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604846 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-credential-socket\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.605026 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604905 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-certs\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.605026 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604950 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istiod-ca-cert\") pod \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\" (UID: \"d2b6b01b-4a53-42db-88de-eb3dec95a0f8\") " Jul 13 22:12:07.605026 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.604950 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:12:07.605490 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.605239 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:12:07.605490 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.605440 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:12:07.605490 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.605455 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:12:07.605721 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.605538 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.605721 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.605558 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.605862 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.605766 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:12:07.607031 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.607000 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-kube-api-access-d9zzm" (OuterVolumeSpecName: "kube-api-access-d9zzm") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "kube-api-access-d9zzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:12:07.607138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.607028 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:12:07.607187 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.607146 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:12:07.607187 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.607153 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2b6b01b-4a53-42db-88de-eb3dec95a0f8" (UID: "d2b6b01b-4a53-42db-88de-eb3dec95a0f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:12:07.706570 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706538 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.706570 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706568 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.706570 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706581 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.706853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706594 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9zzm\" (UniqueName: \"kubernetes.io/projected/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-kube-api-access-d9zzm\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.706853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706606 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.706853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706614 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:07.706853 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:07.706627 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2b6b01b-4a53-42db-88de-eb3dec95a0f8-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:12:08.315188 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.315152 2583 generic.go:356] "Generic (PLEG): container finished" podID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerID="40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312" exitCode=0 Jul 13 22:12:08.315447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.315196 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" event={"ID":"d2b6b01b-4a53-42db-88de-eb3dec95a0f8","Type":"ContainerDied","Data":"40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312"} Jul 13 22:12:08.315447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.315204 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" Jul 13 22:12:08.315447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.315249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n" event={"ID":"d2b6b01b-4a53-42db-88de-eb3dec95a0f8","Type":"ContainerDied","Data":"d01db341777bc4e51bd7893e4fa1dc2aa711803ba03a558b91e8e684db6e3137"} Jul 13 22:12:08.315447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.315278 2583 scope.go:117] "RemoveContainer" containerID="40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312" Jul 13 22:12:08.324923 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.324906 2583 scope.go:117] "RemoveContainer" containerID="78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d" Jul 13 22:12:08.333088 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.333060 2583 scope.go:117] "RemoveContainer" containerID="40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312" Jul 13 22:12:08.333323 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:12:08.333302 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312\": container with ID starting with 40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312 not found: ID does not exist" containerID="40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312" Jul 13 22:12:08.333398 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.333337 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312"} err="failed to get container status \"40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312\": rpc error: code = NotFound desc = could not find container \"40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312\": container with ID starting with 40444e6b1bfab6e52d168bba647656ccb2062ab7e5b39556f62922f647e48312 not found: ID does not exist" Jul 13 22:12:08.333398 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.333362 2583 scope.go:117] "RemoveContainer" containerID="78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d" Jul 13 22:12:08.333632 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:12:08.333609 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d\": container with ID starting with 78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d not found: ID does not exist" containerID="78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d" Jul 13 22:12:08.333676 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.333643 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d"} err="failed to get container status \"78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d\": rpc error: code = NotFound desc = could not find container \"78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d\": container with ID starting with 78ebaf41b29c6b0a8e49695b315985e459d624fe5748d664e01bd8d9f104534d not found: ID does not exist" Jul 13 22:12:08.341542 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.341504 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n"] Jul 13 22:12:08.344689 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.344669 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9dff5a-openshift-default-5d7b896b4c-zbd5n"] Jul 13 22:12:08.359750 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:08.355083 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" path="/var/lib/kubelet/pods/d2b6b01b-4a53-42db-88de-eb3dec95a0f8/volumes" Jul 13 22:12:11.381514 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.381480 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88"] Jul 13 22:12:11.382089 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382069 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" Jul 13 22:12:11.382180 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382090 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" Jul 13 22:12:11.382180 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382105 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" Jul 13 22:12:11.382180 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382114 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" Jul 13 22:12:11.382180 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382130 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" Jul 13 22:12:11.382180 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382140 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" Jul 13 22:12:11.382427 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382241 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="23881415-ce76-42c0-a9f3-638f6bf785d6" containerName="istio-proxy" Jul 13 22:12:11.382427 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382253 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" Jul 13 22:12:11.382427 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.382260 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b6b01b-4a53-42db-88de-eb3dec95a0f8" containerName="istio-proxy" Jul 13 22:12:11.386828 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.386808 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.389578 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.389560 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-d8932b-openshift-default-dockercfg-dqxpk\"" Jul 13 22:12:11.400043 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.400020 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88"] Jul 13 22:12:11.539971 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.539934 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-credential-socket\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.539980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-data\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-socket\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-envoy\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3702671a-1a7b-405c-8d86-a310efecb980-istio-podinfo\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6mq\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-kube-api-access-kv6mq\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540177 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-istio-token\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-certs\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.540447 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.540218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3702671a-1a7b-405c-8d86-a310efecb980-istiod-ca-cert\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.640974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-credential-socket\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641052 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641008 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-data\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641245 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641060 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-socket\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641245 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641096 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-envoy\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641245 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3702671a-1a7b-405c-8d86-a310efecb980-istio-podinfo\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641245 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6mq\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-kube-api-access-kv6mq\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641245 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641214 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-istio-token\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641245 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-certs\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641275 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3702671a-1a7b-405c-8d86-a310efecb980-istiod-ca-cert\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-credential-socket\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641512 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641440 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-data\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641724 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641607 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-certs\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641797 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641756 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-socket\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.641977 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.641959 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3702671a-1a7b-405c-8d86-a310efecb980-istiod-ca-cert\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.643421 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.643394 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-envoy\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.643859 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.643841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3702671a-1a7b-405c-8d86-a310efecb980-istio-podinfo\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.649687 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.649659 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-istio-token\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.649942 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.649924 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6mq\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-kube-api-access-kv6mq\") pod \"e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.697633 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.697605 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:11.836926 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.836899 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88"] Jul 13 22:12:11.839104 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:12:11.839077 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3702671a_1a7b_405c_8d86_a310efecb980.slice/crio-023d8fb86e70d8f4e6090fe2e9d97aacb875e49d983469871ee36d4d40d7de0d WatchSource:0}: Error finding container 023d8fb86e70d8f4e6090fe2e9d97aacb875e49d983469871ee36d4d40d7de0d: Status 404 returned error can't find the container with id 023d8fb86e70d8f4e6090fe2e9d97aacb875e49d983469871ee36d4d40d7de0d Jul 13 22:12:11.841357 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.841323 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:11.841438 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.841391 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:11.841438 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:11.841427 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:12.343743 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:12.343708 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" event={"ID":"3702671a-1a7b-405c-8d86-a310efecb980","Type":"ContainerStarted","Data":"bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901"} Jul 13 22:12:12.343933 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:12.343759 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" event={"ID":"3702671a-1a7b-405c-8d86-a310efecb980","Type":"ContainerStarted","Data":"023d8fb86e70d8f4e6090fe2e9d97aacb875e49d983469871ee36d4d40d7de0d"} Jul 13 22:12:12.368027 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:12.367979 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podStartSLOduration=1.367963647 podStartE2EDuration="1.367963647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:12:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:12:12.36495432 +0000 UTC m=+2553.184464150" watchObservedRunningTime="2026-07-13 22:12:12.367963647 +0000 UTC m=+2553.187473456" Jul 13 22:12:12.698796 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:12.698739 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:12.704095 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:12.704069 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:13.348931 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:13.348902 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:13.349766 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:13.349747 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:19.381150 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:19.381066 2583 generic.go:356] "Generic (PLEG): container finished" podID="361c18b5-3e40-4650-9f2a-94904b96609f" containerID="a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7" exitCode=0 Jul 13 22:12:19.381150 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:19.381120 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" event={"ID":"361c18b5-3e40-4650-9f2a-94904b96609f","Type":"ContainerDied","Data":"a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7"} Jul 13 22:12:19.381694 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:19.381590 2583 scope.go:117] "RemoveContainer" containerID="a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7" Jul 13 22:12:19.382658 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:19.382618 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:19.382755 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:19.382696 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:19.382755 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:19.382732 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:20.298357 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:20.298306 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:20.392804 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:20.392746 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" event={"ID":"361c18b5-3e40-4650-9f2a-94904b96609f","Type":"ContainerStarted","Data":"92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89"} Jul 13 22:12:20.438593 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:20.438561 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:20.439567 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:20.439539 2583 patch_prober.go:28] interesting pod/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 13 22:12:20.439685 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:20.439591 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 13 22:12:21.438534 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:21.438498 2583 patch_prober.go:28] interesting pod/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 13 22:12:21.438921 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:21.438548 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 13 22:12:22.437806 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:22.437744 2583 patch_prober.go:28] interesting pod/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 13 22:12:22.438336 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:22.438014 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 13 22:12:23.438027 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:23.437994 2583 patch_prober.go:28] interesting pod/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 13 22:12:23.438415 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:23.438069 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 13 22:12:24.442273 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:24.442239 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:24.442679 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:24.442588 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:24.443167 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:24.443151 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:12:28.350239 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.350209 2583 patch_prober.go:28] interesting pod/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 13 22:12:28.350656 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.350258 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 13 22:12:28.447725 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.447689 2583 generic.go:356] "Generic (PLEG): container finished" podID="3702671a-1a7b-405c-8d86-a310efecb980" containerID="bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901" exitCode=0 Jul 13 22:12:28.447907 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.447736 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" event={"ID":"3702671a-1a7b-405c-8d86-a310efecb980","Type":"ContainerDied","Data":"bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901"} Jul 13 22:12:28.448141 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.448116 2583 scope.go:117] "RemoveContainer" containerID="bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901" Jul 13 22:12:28.449089 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.449052 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:28.449149 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.449121 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:28.449192 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:28.449154 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 22:12:29.455361 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:29.455325 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" event={"ID":"3702671a-1a7b-405c-8d86-a310efecb980","Type":"ContainerStarted","Data":"2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b"} Jul 13 22:12:29.698535 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:29.698487 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:29.698814 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:29.698764 2583 patch_prober.go:28] interesting pod/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 13 22:12:29.698879 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:29.698849 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 13 22:12:30.698193 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:30.698163 2583 patch_prober.go:28] interesting pod/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 13 22:12:30.699903 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:30.698226 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 13 22:12:31.698788 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:31.698726 2583 patch_prober.go:28] interesting pod/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 13 22:12:31.699190 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:31.698821 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 13 22:12:32.698418 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:32.698381 2583 patch_prober.go:28] interesting pod/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 13 22:12:32.698637 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:32.698442 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 13 22:12:33.701719 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:33.701686 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:33.702119 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:33.702026 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:12:33.702646 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:12:33.702630 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:14:40.666154 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:14:40.666079 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:14:40.683970 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:14:40.683946 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:18:02.630291 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.630210 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:02.633545 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.633527 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:02.636180 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.636154 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6g8h9\"" Jul 13 22:18:02.639116 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.639088 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:02.712045 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.712010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6c8v\" (UniqueName: \"kubernetes.io/projected/35a4ff71-9266-4e03-ab77-62a25000ce61-kube-api-access-r6c8v\") pod \"test-curl-7605-139927600975920\" (UID: \"35a4ff71-9266-4e03-ab77-62a25000ce61\") " pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:02.813131 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.813095 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6c8v\" (UniqueName: \"kubernetes.io/projected/35a4ff71-9266-4e03-ab77-62a25000ce61-kube-api-access-r6c8v\") pod \"test-curl-7605-139927600975920\" (UID: \"35a4ff71-9266-4e03-ab77-62a25000ce61\") " pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:02.821328 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.821292 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6c8v\" (UniqueName: \"kubernetes.io/projected/35a4ff71-9266-4e03-ab77-62a25000ce61-kube-api-access-r6c8v\") pod \"test-curl-7605-139927600975920\" (UID: \"35a4ff71-9266-4e03-ab77-62a25000ce61\") " pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:02.943222 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:02.943191 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:03.079822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:03.079794 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:03.081616 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:18:03.081588 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35a4ff71_9266_4e03_ab77_62a25000ce61.slice/crio-1c5dda75db2e986b24011bc34a2bcdc311d249a4a74fab5777e0ed7652939014 WatchSource:0}: Error finding container 1c5dda75db2e986b24011bc34a2bcdc311d249a4a74fab5777e0ed7652939014: Status 404 returned error can't find the container with id 1c5dda75db2e986b24011bc34a2bcdc311d249a4a74fab5777e0ed7652939014 Jul 13 22:18:03.082920 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:03.082896 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:18:03.277210 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:03.277110 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600975920" event={"ID":"35a4ff71-9266-4e03-ab77-62a25000ce61","Type":"ContainerStarted","Data":"1c5dda75db2e986b24011bc34a2bcdc311d249a4a74fab5777e0ed7652939014"} Jul 13 22:18:04.284590 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:04.284551 2583 generic.go:356] "Generic (PLEG): container finished" podID="35a4ff71-9266-4e03-ab77-62a25000ce61" containerID="4562e8921ea6dbd1d57191bb99527b71ca44aef4f971e7a0028f346c85129c26" exitCode=0 Jul 13 22:18:04.285117 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:04.284600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600975920" event={"ID":"35a4ff71-9266-4e03-ab77-62a25000ce61","Type":"ContainerDied","Data":"4562e8921ea6dbd1d57191bb99527b71ca44aef4f971e7a0028f346c85129c26"} Jul 13 22:18:05.426355 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.426328 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:05.460199 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.460163 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7605-139927600975920_35a4ff71-9266-4e03-ab77-62a25000ce61/test-curl-7605-139927600975920/0.log" Jul 13 22:18:05.497947 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.497904 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:05.505763 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.505733 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:05.537659 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.537627 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6c8v\" (UniqueName: \"kubernetes.io/projected/35a4ff71-9266-4e03-ab77-62a25000ce61-kube-api-access-r6c8v\") pod \"35a4ff71-9266-4e03-ab77-62a25000ce61\" (UID: \"35a4ff71-9266-4e03-ab77-62a25000ce61\") " Jul 13 22:18:05.539666 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.539643 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35a4ff71-9266-4e03-ab77-62a25000ce61-kube-api-access-r6c8v" (OuterVolumeSpecName: "kube-api-access-r6c8v") pod "35a4ff71-9266-4e03-ab77-62a25000ce61" (UID: "35a4ff71-9266-4e03-ab77-62a25000ce61"). InnerVolumeSpecName "kube-api-access-r6c8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:18:05.624505 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.624472 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:05.624971 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.624956 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a4ff71-9266-4e03-ab77-62a25000ce61" containerName="test-curl-7605-139927600975920" Jul 13 22:18:05.625026 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.624972 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a4ff71-9266-4e03-ab77-62a25000ce61" containerName="test-curl-7605-139927600975920" Jul 13 22:18:05.625062 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.625030 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a4ff71-9266-4e03-ab77-62a25000ce61" containerName="test-curl-7605-139927600975920" Jul 13 22:18:05.628150 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.628130 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:05.635079 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.635053 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:05.638520 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.638499 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6c8v\" (UniqueName: \"kubernetes.io/projected/35a4ff71-9266-4e03-ab77-62a25000ce61-kube-api-access-r6c8v\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:05.739656 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.739627 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2jdq\" (UniqueName: \"kubernetes.io/projected/7bb08ecd-4c98-4318-a6d8-41278361d3c8-kube-api-access-l2jdq\") pod \"test-curl-7605-139927600975920\" (UID: \"7bb08ecd-4c98-4318-a6d8-41278361d3c8\") " pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:05.841306 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.841211 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2jdq\" (UniqueName: \"kubernetes.io/projected/7bb08ecd-4c98-4318-a6d8-41278361d3c8-kube-api-access-l2jdq\") pod \"test-curl-7605-139927600975920\" (UID: \"7bb08ecd-4c98-4318-a6d8-41278361d3c8\") " pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:05.850315 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.850288 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2jdq\" (UniqueName: \"kubernetes.io/projected/7bb08ecd-4c98-4318-a6d8-41278361d3c8-kube-api-access-l2jdq\") pod \"test-curl-7605-139927600975920\" (UID: \"7bb08ecd-4c98-4318-a6d8-41278361d3c8\") " pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:05.938966 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:05.938928 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:06.073854 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:06.073827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:06.075533 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:18:06.075507 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bb08ecd_4c98_4318_a6d8_41278361d3c8.slice/crio-228cf6fff1d155867120b4bc9e5c2c89cb97f1cfffe396df680011007e753796 WatchSource:0}: Error finding container 228cf6fff1d155867120b4bc9e5c2c89cb97f1cfffe396df680011007e753796: Status 404 returned error can't find the container with id 228cf6fff1d155867120b4bc9e5c2c89cb97f1cfffe396df680011007e753796 Jul 13 22:18:06.297172 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:06.297140 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c5dda75db2e986b24011bc34a2bcdc311d249a4a74fab5777e0ed7652939014" Jul 13 22:18:06.297172 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:06.297142 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:06.298282 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:06.298256 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600975920" event={"ID":"7bb08ecd-4c98-4318-a6d8-41278361d3c8","Type":"ContainerStarted","Data":"228cf6fff1d155867120b4bc9e5c2c89cb97f1cfffe396df680011007e753796"} Jul 13 22:18:06.308517 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:06.308486 2583 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7605-139927600975920" oldPodUID="35a4ff71-9266-4e03-ab77-62a25000ce61" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" Jul 13 22:18:06.352750 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:06.352672 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35a4ff71-9266-4e03-ab77-62a25000ce61" path="/var/lib/kubelet/pods/35a4ff71-9266-4e03-ab77-62a25000ce61/volumes" Jul 13 22:18:07.305659 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:07.305623 2583 generic.go:356] "Generic (PLEG): container finished" podID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" containerID="6d2809715eea0df7702b4c5e795cedd1338e71aaea31d9c167b9f4c0cd072963" exitCode=0 Jul 13 22:18:07.306083 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:07.305713 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600975920" event={"ID":"7bb08ecd-4c98-4318-a6d8-41278361d3c8","Type":"ContainerDied","Data":"6d2809715eea0df7702b4c5e795cedd1338e71aaea31d9c167b9f4c0cd072963"} Jul 13 22:18:08.450893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.450864 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:08.469173 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.469146 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7605-139927600975920_7bb08ecd-4c98-4318-a6d8-41278361d3c8/test-curl-7605-139927600975920/0.log" Jul 13 22:18:08.490081 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.490055 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:08.493461 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.493442 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-139927600975920"] Jul 13 22:18:08.569063 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.569009 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2jdq\" (UniqueName: \"kubernetes.io/projected/7bb08ecd-4c98-4318-a6d8-41278361d3c8-kube-api-access-l2jdq\") pod \"7bb08ecd-4c98-4318-a6d8-41278361d3c8\" (UID: \"7bb08ecd-4c98-4318-a6d8-41278361d3c8\") " Jul 13 22:18:08.570898 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.570868 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb08ecd-4c98-4318-a6d8-41278361d3c8-kube-api-access-l2jdq" (OuterVolumeSpecName: "kube-api-access-l2jdq") pod "7bb08ecd-4c98-4318-a6d8-41278361d3c8" (UID: "7bb08ecd-4c98-4318-a6d8-41278361d3c8"). InnerVolumeSpecName "kube-api-access-l2jdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:18:08.670063 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:08.670023 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2jdq\" (UniqueName: \"kubernetes.io/projected/7bb08ecd-4c98-4318-a6d8-41278361d3c8-kube-api-access-l2jdq\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:09.318259 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:09.318231 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:09.318454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:09.318272 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="228cf6fff1d155867120b4bc9e5c2c89cb97f1cfffe396df680011007e753796" Jul 13 22:18:09.328296 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:18:09.328271 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-139927600975920\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:10.352830 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:18:10.352791 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-139927600975920\" is forbidden: User \"system:node:ip-10-0-133-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-19.ec2.internal' and this object" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" pod="opendatahub/test-curl-7605-139927600975920" Jul 13 22:18:10.353201 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:10.353094 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" path="/var/lib/kubelet/pods/7bb08ecd-4c98-4318-a6d8-41278361d3c8/volumes" Jul 13 22:18:40.493579 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:40.493543 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd"] Jul 13 22:18:40.494072 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:40.493809 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" containerID="cri-o://92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89" gracePeriod=30 Jul 13 22:18:45.743455 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.743431 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:18:45.778241 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778185 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-certs\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778241 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778223 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-envoy\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778425 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778257 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-socket\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778425 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778296 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flpq7\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-kube-api-access-flpq7\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778425 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778323 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-data\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778425 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778340 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-credential-socket\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778425 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778396 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/361c18b5-3e40-4650-9f2a-94904b96609f-istio-podinfo\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778724 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778427 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/361c18b5-3e40-4650-9f2a-94904b96609f-istiod-ca-cert\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778500 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:18:45.778894 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778605 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:18:45.778894 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778686 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:18:45.778997 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778916 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-istio-token\") pod \"361c18b5-3e40-4650-9f2a-94904b96609f\" (UID: \"361c18b5-3e40-4650-9f2a-94904b96609f\") " Jul 13 22:18:45.778997 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.778928 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/361c18b5-3e40-4650-9f2a-94904b96609f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:18:45.779245 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.779228 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/361c18b5-3e40-4650-9f2a-94904b96609f-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.779315 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.779248 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.779315 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.779262 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.779315 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.779273 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.779474 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.779338 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-data" (OuterVolumeSpecName: "istio-data") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:18:45.780515 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.780489 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:18:45.780611 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.780492 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-kube-api-access-flpq7" (OuterVolumeSpecName: "kube-api-access-flpq7") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "kube-api-access-flpq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:18:45.780962 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.780941 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/361c18b5-3e40-4650-9f2a-94904b96609f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:18:45.781126 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.781107 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-istio-token" (OuterVolumeSpecName: "istio-token") pod "361c18b5-3e40-4650-9f2a-94904b96609f" (UID: "361c18b5-3e40-4650-9f2a-94904b96609f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:18:45.879878 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.879856 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.879878 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.879878 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flpq7\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-kube-api-access-flpq7\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.880003 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.879888 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/361c18b5-3e40-4650-9f2a-94904b96609f-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.880003 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.879898 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/361c18b5-3e40-4650-9f2a-94904b96609f-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:45.880003 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:45.879907 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/361c18b5-3e40-4650-9f2a-94904b96609f-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:18:46.519582 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.519556 2583 generic.go:356] "Generic (PLEG): container finished" podID="361c18b5-3e40-4650-9f2a-94904b96609f" containerID="92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89" exitCode=0 Jul 13 22:18:46.519732 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.519608 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" Jul 13 22:18:46.519732 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.519625 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" event={"ID":"361c18b5-3e40-4650-9f2a-94904b96609f","Type":"ContainerDied","Data":"92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89"} Jul 13 22:18:46.519732 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.519682 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd" event={"ID":"361c18b5-3e40-4650-9f2a-94904b96609f","Type":"ContainerDied","Data":"99845e5e3e4d678721708639a36fb1ebdffad8417336c87579097320fd0b945e"} Jul 13 22:18:46.519732 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.519712 2583 scope.go:117] "RemoveContainer" containerID="92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89" Jul 13 22:18:46.528830 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.528814 2583 scope.go:117] "RemoveContainer" containerID="a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7" Jul 13 22:18:46.540458 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.540359 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd"] Jul 13 22:18:46.540537 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.540476 2583 scope.go:117] "RemoveContainer" containerID="92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89" Jul 13 22:18:46.540810 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:18:46.540757 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89\": container with ID starting with 92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89 not found: ID does not exist" containerID="92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89" Jul 13 22:18:46.540888 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.540814 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89"} err="failed to get container status \"92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89\": rpc error: code = NotFound desc = could not find container \"92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89\": container with ID starting with 92dd8f9b59aefac6dbed7fa56d6633b871b98420cead6ad711f06462f0c2ba89 not found: ID does not exist" Jul 13 22:18:46.540888 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.540840 2583 scope.go:117] "RemoveContainer" containerID="a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7" Jul 13 22:18:46.548650 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:18:46.541465 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7\": container with ID starting with a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7 not found: ID does not exist" containerID="a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7" Jul 13 22:18:46.548650 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.541502 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7"} err="failed to get container status \"a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7\": rpc error: code = NotFound desc = could not find container \"a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7\": container with ID starting with a9f32f727c5ad6e30fcc8c6c12a890028eb1d1fe51f59a4e4d9d34d4c8d017b7 not found: ID does not exist" Jul 13 22:18:46.548650 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:46.545621 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f89343-openshift-default-75fdc7db79-6sgbd"] Jul 13 22:18:48.353769 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:18:48.353737 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" path="/var/lib/kubelet/pods/361c18b5-3e40-4650-9f2a-94904b96609f/volumes" Jul 13 22:19:06.620441 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:06.620406 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88"] Jul 13 22:19:06.620994 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:06.620629 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" containerID="cri-o://2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b" gracePeriod=30 Jul 13 22:19:07.317912 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:07.317882 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b79df98b-9xfwr_2eba5936-4553-450f-a11a-443c14742094/authorino/0.log" Jul 13 22:19:11.873168 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.873146 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:19:11.986737 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.986714 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-data\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.986902 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.986756 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-certs\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.986902 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.986789 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3702671a-1a7b-405c-8d86-a310efecb980-istiod-ca-cert\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.986902 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.986812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv6mq\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-kube-api-access-kv6mq\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.986902 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.986882 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-socket\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.987138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987022 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-envoy\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.987138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987052 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:19:11.987138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987070 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-credential-socket\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.987138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987099 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3702671a-1a7b-405c-8d86-a310efecb980-istio-podinfo\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.987138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987130 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-istio-token\") pod \"3702671a-1a7b-405c-8d86-a310efecb980\" (UID: \"3702671a-1a7b-405c-8d86-a310efecb980\") " Jul 13 22:19:11.987404 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987155 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3702671a-1a7b-405c-8d86-a310efecb980-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:19:11.987404 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987178 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:19:11.987404 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987346 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:19:11.987404 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987402 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:11.987615 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987418 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-credential-socket\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:11.987615 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987431 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-workload-certs\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:11.987615 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987444 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3702671a-1a7b-405c-8d86-a310efecb980-istiod-ca-cert\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:11.987790 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.987702 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-data" (OuterVolumeSpecName: "istio-data") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:19:11.989037 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.989010 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-kube-api-access-kv6mq" (OuterVolumeSpecName: "kube-api-access-kv6mq") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "kube-api-access-kv6mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:19:11.989138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.989043 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:19:11.989138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.989117 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-istio-token" (OuterVolumeSpecName: "istio-token") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:19:11.989138 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:11.989124 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3702671a-1a7b-405c-8d86-a310efecb980-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3702671a-1a7b-405c-8d86-a310efecb980" (UID: "3702671a-1a7b-405c-8d86-a310efecb980"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 22:19:12.088635 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.088609 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-envoy\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:12.088635 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.088631 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3702671a-1a7b-405c-8d86-a310efecb980-istio-podinfo\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:12.088635 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.088639 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-istio-token\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:12.088828 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.088647 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3702671a-1a7b-405c-8d86-a310efecb980-istio-data\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:12.088828 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.088655 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv6mq\" (UniqueName: \"kubernetes.io/projected/3702671a-1a7b-405c-8d86-a310efecb980-kube-api-access-kv6mq\") on node \"ip-10-0-133-19.ec2.internal\" DevicePath \"\"" Jul 13 22:19:12.214626 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.214597 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-ptxzb_f41427fd-dc84-4aec-aeb7-2a70a840f081/manager/0.log" Jul 13 22:19:12.336618 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.336554 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hjj6p_a108cde0-a23b-46fc-81a7-85d22163d74e/manager/0.log" Jul 13 22:19:12.578401 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.578374 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9swdn_cd2ea4b2-0027-4cd7-8e29-e36db1367a36/server/0.log" Jul 13 22:19:12.664307 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.664221 2583 generic.go:356] "Generic (PLEG): container finished" podID="3702671a-1a7b-405c-8d86-a310efecb980" containerID="2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b" exitCode=0 Jul 13 22:19:12.664307 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.664272 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" Jul 13 22:19:12.664307 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.664297 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" event={"ID":"3702671a-1a7b-405c-8d86-a310efecb980","Type":"ContainerDied","Data":"2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b"} Jul 13 22:19:12.664626 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.664346 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88" event={"ID":"3702671a-1a7b-405c-8d86-a310efecb980","Type":"ContainerDied","Data":"023d8fb86e70d8f4e6090fe2e9d97aacb875e49d983469871ee36d4d40d7de0d"} Jul 13 22:19:12.664626 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.664369 2583 scope.go:117] "RemoveContainer" containerID="2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b" Jul 13 22:19:12.673461 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.673446 2583 scope.go:117] "RemoveContainer" containerID="bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901" Jul 13 22:19:12.681445 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.681428 2583 scope.go:117] "RemoveContainer" containerID="2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b" Jul 13 22:19:12.681700 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:19:12.681680 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b\": container with ID starting with 2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b not found: ID does not exist" containerID="2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b" Jul 13 22:19:12.681760 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.681710 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b"} err="failed to get container status \"2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b\": rpc error: code = NotFound desc = could not find container \"2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b\": container with ID starting with 2c83a9232756606cb2c78af8497ddc7572b4ec5698850b802f45260ca55dfe3b not found: ID does not exist" Jul 13 22:19:12.681760 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.681728 2583 scope.go:117] "RemoveContainer" containerID="bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901" Jul 13 22:19:12.682029 ip-10-0-133-19 kubenswrapper[2583]: E0713 22:19:12.682005 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901\": container with ID starting with bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901 not found: ID does not exist" containerID="bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901" Jul 13 22:19:12.682091 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.682049 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901"} err="failed to get container status \"bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901\": rpc error: code = NotFound desc = could not find container \"bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901\": container with ID starting with bad25224ecf4a072d1ec26ddfd23b983c9a74d285af3e60318137d072f438901 not found: ID does not exist" Jul 13 22:19:12.683183 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.683167 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88"] Jul 13 22:19:12.686758 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.686738 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-d8932b-openshift-default-6dcf6774bc-wtv88"] Jul 13 22:19:12.693822 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.693803 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-57hdq_7d3a8120-670f-41a4-bfcb-1cce715a150b/manager/0.log" Jul 13 22:19:12.932386 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.932361 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-728pc_06e0de3a-b207-4d77-b51c-2e05e01b3b5c/copy-manifests/0.log" Jul 13 22:19:12.944550 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:12.944526 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-728pc_06e0de3a-b207-4d77-b51c-2e05e01b3b5c/manager/0.log" Jul 13 22:19:13.437176 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:13.437134 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5b9df4b459-78qps_02d0a6cc-4924-4ba7-a5a0-3daacca69fd3/maas-api/0.log" Jul 13 22:19:14.352184 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:14.352148 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3702671a-1a7b-405c-8d86-a310efecb980" path="/var/lib/kubelet/pods/3702671a-1a7b-405c-8d86-a310efecb980/volumes" Jul 13 22:19:15.329414 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:15.329380 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5r5hk_18b55dab-ad2f-4ab7-a464-b1f7be2052ae/postgres/0.log" Jul 13 22:19:16.118868 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.118838 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m_07b95b97-1c92-43b0-adf3-700bf6dc7c22/util/0.log" Jul 13 22:19:16.124614 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.124595 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m_07b95b97-1c92-43b0-adf3-700bf6dc7c22/pull/0.log" Jul 13 22:19:16.130258 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.130239 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m_07b95b97-1c92-43b0-adf3-700bf6dc7c22/extract/0.log" Jul 13 22:19:16.245598 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.245575 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l_f8fe055a-6f53-408a-8d90-cfad5acc4780/util/0.log" Jul 13 22:19:16.251440 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.251421 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l_f8fe055a-6f53-408a-8d90-cfad5acc4780/pull/0.log" Jul 13 22:19:16.257212 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.257192 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l_f8fe055a-6f53-408a-8d90-cfad5acc4780/extract/0.log" Jul 13 22:19:16.370825 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.370754 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw_b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73/util/0.log" Jul 13 22:19:16.376712 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.376695 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw_b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73/pull/0.log" Jul 13 22:19:16.382309 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.382290 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw_b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73/extract/0.log" Jul 13 22:19:16.499235 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.499217 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv_56237d87-337e-4ac6-8202-fd631a54c130/util/0.log" Jul 13 22:19:16.505068 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.505052 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv_56237d87-337e-4ac6-8202-fd631a54c130/pull/0.log" Jul 13 22:19:16.510604 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.510589 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv_56237d87-337e-4ac6-8202-fd631a54c130/extract/0.log" Jul 13 22:19:16.635715 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.635661 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b79df98b-9xfwr_2eba5936-4553-450f-a11a-443c14742094/authorino/0.log" Jul 13 22:19:16.759624 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.759598 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fs8q5_387b0370-a3aa-495a-b6a6-36be6b94ba47/manager/0.log" Jul 13 22:19:16.989169 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:16.989144 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4s9mw_02ef0294-96c7-45be-94d4-76d120d6e6b1/kuadrant-console-plugin/0.log" Jul 13 22:19:17.108814 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:17.108788 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t99mn_3f71e7f6-29b6-44b2-b2a2-18332f105064/registry-server/0.log" Jul 13 22:19:17.474088 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:17.474066 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zk7sj_68445109-b151-408b-8ec4-538bb7cae217/manager/0.log" Jul 13 22:19:17.827025 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:17.826954 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4npv5l_95af6daa-68a4-4861-95d7-40c4ee1c761a/istio-proxy/0.log" Jul 13 22:19:18.178334 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:18.178313 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-kh9fs_3d654a0c-921f-475d-b656-2795bd12952a/kube-auth-proxy/0.log" Jul 13 22:19:18.302012 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:18.301987 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/1.log" Jul 13 22:19:19.021032 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:19.020993 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-n4rdf_809238fe-6b36-4077-9b43-361145c555d3/main/0.log" Jul 13 22:19:19.383329 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:19.383245 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfdxmm_bb2474b8-3eac-4c7c-a4bd-81a51afa3ca8/main/0.log" Jul 13 22:19:23.568068 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568035 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wjrg5/must-gather-qrm82"] Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568378 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568389 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568396 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568402 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568416 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568421 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568436 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" containerName="test-curl-7605-139927600975920" Jul 13 22:19:23.568454 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568442 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" containerName="test-curl-7605-139927600975920" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568492 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb08ecd-4c98-4318-a6d8-41278361d3c8" containerName="test-curl-7605-139927600975920" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568499 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568507 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568513 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="361c18b5-3e40-4650-9f2a-94904b96609f" containerName="istio-proxy" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568576 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568582 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" Jul 13 22:19:23.568764 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.568667 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3702671a-1a7b-405c-8d86-a310efecb980" containerName="istio-proxy" Jul 13 22:19:23.571644 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.571624 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.573989 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.573970 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wjrg5\"/\"openshift-service-ca.crt\"" Jul 13 22:19:23.575080 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.575060 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wjrg5\"/\"default-dockercfg-wn26n\"" Jul 13 22:19:23.575185 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.575088 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wjrg5\"/\"kube-root-ca.crt\"" Jul 13 22:19:23.581126 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.581105 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjrg5/must-gather-qrm82"] Jul 13 22:19:23.677877 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.677853 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7c4d013f-2ee0-41ed-ac4e-a833c84b1692-must-gather-output\") pod \"must-gather-qrm82\" (UID: \"7c4d013f-2ee0-41ed-ac4e-a833c84b1692\") " pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.677993 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.677890 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5n4s\" (UniqueName: \"kubernetes.io/projected/7c4d013f-2ee0-41ed-ac4e-a833c84b1692-kube-api-access-l5n4s\") pod \"must-gather-qrm82\" (UID: \"7c4d013f-2ee0-41ed-ac4e-a833c84b1692\") " pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.778997 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.778967 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7c4d013f-2ee0-41ed-ac4e-a833c84b1692-must-gather-output\") pod \"must-gather-qrm82\" (UID: \"7c4d013f-2ee0-41ed-ac4e-a833c84b1692\") " pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.779108 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.779004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5n4s\" (UniqueName: \"kubernetes.io/projected/7c4d013f-2ee0-41ed-ac4e-a833c84b1692-kube-api-access-l5n4s\") pod \"must-gather-qrm82\" (UID: \"7c4d013f-2ee0-41ed-ac4e-a833c84b1692\") " pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.779285 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.779265 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7c4d013f-2ee0-41ed-ac4e-a833c84b1692-must-gather-output\") pod \"must-gather-qrm82\" (UID: \"7c4d013f-2ee0-41ed-ac4e-a833c84b1692\") " pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.787643 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.787619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5n4s\" (UniqueName: \"kubernetes.io/projected/7c4d013f-2ee0-41ed-ac4e-a833c84b1692-kube-api-access-l5n4s\") pod \"must-gather-qrm82\" (UID: \"7c4d013f-2ee0-41ed-ac4e-a833c84b1692\") " pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:23.881408 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:23.881343 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjrg5/must-gather-qrm82" Jul 13 22:19:24.009309 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:24.009285 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjrg5/must-gather-qrm82"] Jul 13 22:19:24.011048 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:19:24.011025 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c4d013f_2ee0_41ed_ac4e_a833c84b1692.slice/crio-589ab9b08ad4091a966b0e6433748ac6f17b77751773e0e3a591fdcce0adc5bd WatchSource:0}: Error finding container 589ab9b08ad4091a966b0e6433748ac6f17b77751773e0e3a591fdcce0adc5bd: Status 404 returned error can't find the container with id 589ab9b08ad4091a966b0e6433748ac6f17b77751773e0e3a591fdcce0adc5bd Jul 13 22:19:24.734550 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:24.734504 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/must-gather-qrm82" event={"ID":"7c4d013f-2ee0-41ed-ac4e-a833c84b1692","Type":"ContainerStarted","Data":"589ab9b08ad4091a966b0e6433748ac6f17b77751773e0e3a591fdcce0adc5bd"} Jul 13 22:19:25.745428 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:25.745390 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/must-gather-qrm82" event={"ID":"7c4d013f-2ee0-41ed-ac4e-a833c84b1692","Type":"ContainerStarted","Data":"0dc8ce8c3e0bd314c3de8fdd90a45f002c61ce66fc463cb8a6175f11949c8cd5"} Jul 13 22:19:25.746493 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:25.746468 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/must-gather-qrm82" event={"ID":"7c4d013f-2ee0-41ed-ac4e-a833c84b1692","Type":"ContainerStarted","Data":"48a1eefcd94eacb68c9afe8061b9313db5653719a6c50002b221a0264596a217"} Jul 13 22:19:26.398624 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:26.398589 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ckzb9_4f589797-94ec-4487-b525-157b6547a3d5/global-pull-secret-syncer/0.log" Jul 13 22:19:26.597215 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:26.597180 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tklwp_f8e4813b-1496-4f6e-bc32-5d875132c8cb/konnectivity-agent/0.log" Jul 13 22:19:26.617567 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:26.617539 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-19.ec2.internal_052647b0243d11ac5a4990de0eb190ae/haproxy/0.log" Jul 13 22:19:30.686155 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.686127 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m_07b95b97-1c92-43b0-adf3-700bf6dc7c22/extract/0.log" Jul 13 22:19:30.711231 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.711158 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m_07b95b97-1c92-43b0-adf3-700bf6dc7c22/util/0.log" Jul 13 22:19:30.737520 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.737480 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fl45m_07b95b97-1c92-43b0-adf3-700bf6dc7c22/pull/0.log" Jul 13 22:19:30.767526 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.767499 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l_f8fe055a-6f53-408a-8d90-cfad5acc4780/extract/0.log" Jul 13 22:19:30.792470 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.792442 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l_f8fe055a-6f53-408a-8d90-cfad5acc4780/util/0.log" Jul 13 22:19:30.814563 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.814536 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw42l_f8fe055a-6f53-408a-8d90-cfad5acc4780/pull/0.log" Jul 13 22:19:30.846457 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.846350 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw_b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73/extract/0.log" Jul 13 22:19:30.871430 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.871404 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw_b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73/util/0.log" Jul 13 22:19:30.901489 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.901466 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734bgxw_b7a06cd3-d24b-4a2c-b4b0-0d38d966bc73/pull/0.log" Jul 13 22:19:30.930382 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.930327 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv_56237d87-337e-4ac6-8202-fd631a54c130/extract/0.log" Jul 13 22:19:30.955151 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.955071 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv_56237d87-337e-4ac6-8202-fd631a54c130/util/0.log" Jul 13 22:19:30.978875 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:30.978822 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bbdnv_56237d87-337e-4ac6-8202-fd631a54c130/pull/0.log" Jul 13 22:19:31.372836 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:31.372514 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b79df98b-9xfwr_2eba5936-4553-450f-a11a-443c14742094/authorino/0.log" Jul 13 22:19:31.402336 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:31.402288 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fs8q5_387b0370-a3aa-495a-b6a6-36be6b94ba47/manager/0.log" Jul 13 22:19:31.452861 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:31.452822 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4s9mw_02ef0294-96c7-45be-94d4-76d120d6e6b1/kuadrant-console-plugin/0.log" Jul 13 22:19:31.500753 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:31.500691 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t99mn_3f71e7f6-29b6-44b2-b2a2-18332f105064/registry-server/0.log" Jul 13 22:19:31.821186 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:31.821128 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zk7sj_68445109-b151-408b-8ec4-538bb7cae217/manager/0.log" Jul 13 22:19:33.623532 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.623392 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-q4svq_11d8cb6a-c43e-4c00-a04e-15eb927c8b0f/monitoring-plugin/0.log" Jul 13 22:19:33.739740 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.739647 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv6jn_3124177d-dc55-454c-b9c9-7a2f3557d4f5/node-exporter/0.log" Jul 13 22:19:33.762377 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.762347 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv6jn_3124177d-dc55-454c-b9c9-7a2f3557d4f5/kube-rbac-proxy/0.log" Jul 13 22:19:33.784663 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.784630 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv6jn_3124177d-dc55-454c-b9c9-7a2f3557d4f5/init-textfile/0.log" Jul 13 22:19:33.888749 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.888695 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-fwhsj_ae395e51-c106-4727-9615-226822487945/kube-rbac-proxy-main/0.log" Jul 13 22:19:33.913077 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.913047 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-fwhsj_ae395e51-c106-4727-9615-226822487945/kube-rbac-proxy-self/0.log" Jul 13 22:19:33.936818 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:33.936788 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-fwhsj_ae395e51-c106-4727-9615-226822487945/openshift-state-metrics/0.log" Jul 13 22:19:34.228582 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:34.228544 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dd44455bd-45775_2aa3e973-b1c9-48cb-8f14-7b879f22ec4c/telemeter-client/0.log" Jul 13 22:19:34.251880 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:34.251843 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dd44455bd-45775_2aa3e973-b1c9-48cb-8f14-7b879f22ec4c/reload/0.log" Jul 13 22:19:34.276269 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:34.276241 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dd44455bd-45775_2aa3e973-b1c9-48cb-8f14-7b879f22ec4c/kube-rbac-proxy/0.log" Jul 13 22:19:36.099275 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.099200 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wjrg5/must-gather-qrm82" podStartSLOduration=12.191630896 podStartE2EDuration="13.099178191s" totalImagesPullingTime="907.547295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:19:25.764176579 +0000 UTC m=+2986.583686386" watchObservedRunningTime="2026-07-13 22:19:36.099178191 +0000 UTC m=+2996.918688000" Jul 13 22:19:36.100503 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.100476 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6"] Jul 13 22:19:36.108274 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.108251 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.113634 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.113604 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6"] Jul 13 22:19:36.303132 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.303091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-podres\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.303442 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.303420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-sys\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.303658 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.303639 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-proc\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.303834 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.303815 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-lib-modules\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.303971 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.303955 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvbsr\" (UniqueName: \"kubernetes.io/projected/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-kube-api-access-bvbsr\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.405165 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.405081 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-podres\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.405573 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.405556 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-sys\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.405850 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.405835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-proc\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.406072 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.406058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-lib-modules\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.406290 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.406274 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvbsr\" (UniqueName: \"kubernetes.io/projected/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-kube-api-access-bvbsr\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.406725 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.405707 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-sys\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.406866 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.405466 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-podres\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.406980 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.406233 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-lib-modules\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.407075 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.405985 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-proc\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.414551 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.414505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvbsr\" (UniqueName: \"kubernetes.io/projected/4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1-kube-api-access-bvbsr\") pod \"perf-node-gather-daemonset-nrbc6\" (UID: \"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.424286 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.424267 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.618256 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.617980 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6"] Jul 13 22:19:36.619559 ip-10-0-133-19 kubenswrapper[2583]: W0713 22:19:36.619531 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4b6ee02b_d2b4_4f71_9113_aaeb8c13a0e1.slice/crio-f22d9bc59b230323681b271fbc7c1aacfead4c67233b98f3fb22f544543f05c1 WatchSource:0}: Error finding container f22d9bc59b230323681b271fbc7c1aacfead4c67233b98f3fb22f544543f05c1: Status 404 returned error can't find the container with id f22d9bc59b230323681b271fbc7c1aacfead4c67233b98f3fb22f544543f05c1 Jul 13 22:19:36.817572 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.817538 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-7l94x_57d45916-3720-410a-a967-c448673c7233/download-server/0.log" Jul 13 22:19:36.851073 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.851035 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" event={"ID":"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1","Type":"ContainerStarted","Data":"f4f68207a171089dfa4fd882a79cc0e6aca2595d33d04c11d026c3e2c8bcfa34"} Jul 13 22:19:36.851263 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.851097 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" event={"ID":"4b6ee02b-d2b4-4f71-9113-aaeb8c13a0e1","Type":"ContainerStarted","Data":"f22d9bc59b230323681b271fbc7c1aacfead4c67233b98f3fb22f544543f05c1"} Jul 13 22:19:36.851337 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.851265 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:36.869901 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:36.869849 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" podStartSLOduration=0.869831384 podStartE2EDuration="869.831384ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 22:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:19:36.867324901 +0000 UTC m=+2997.686834711" watchObservedRunningTime="2026-07-13 22:19:36.869831384 +0000 UTC m=+2997.689341192" Jul 13 22:19:38.329071 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:38.329046 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4ddkh_de0238bd-168e-48aa-9f7c-d8cdea513774/dns/0.log" Jul 13 22:19:38.350893 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:38.350867 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4ddkh_de0238bd-168e-48aa-9f7c-d8cdea513774/kube-rbac-proxy/0.log" Jul 13 22:19:38.522290 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:38.522260 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhlcp_b0438b82-c3d6-4ce8-8d45-fb3ac41c659c/dns-node-resolver/0.log" Jul 13 22:19:39.077302 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:39.077273 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dlbfr_6732ff06-54b6-4879-9977-4bc85ce107da/node-ca/0.log" Jul 13 22:19:40.018652 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:40.018616 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4npv5l_95af6daa-68a4-4861-95d7-40c4ee1c761a/istio-proxy/0.log" Jul 13 22:19:40.190591 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:40.190556 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-kh9fs_3d654a0c-921f-475d-b656-2795bd12952a/kube-auth-proxy/0.log" Jul 13 22:19:40.220544 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:40.220516 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/0.log" Jul 13 22:19:40.257588 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:40.257552 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/1.log" Jul 13 22:19:40.735688 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:40.735650 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:19:40.747291 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:40.747057 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb7hh_f6839532-b922-48be-8590-c4dab166cc5a/ovn-acl-logging/0.log" Jul 13 22:19:41.072009 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:41.071932 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fjmwk_5192b507-e857-4d28-bd49-ec838b6d1664/serve-healthcheck-canary/0.log" Jul 13 22:19:41.687092 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:41.687058 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vm95p_6b8c7d68-2b91-48fd-9a1a-403884ff3f53/kube-rbac-proxy/0.log" Jul 13 22:19:41.708890 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:41.708862 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vm95p_6b8c7d68-2b91-48fd-9a1a-403884ff3f53/exporter/0.log" Jul 13 22:19:41.731794 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:41.731744 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vm95p_6b8c7d68-2b91-48fd-9a1a-403884ff3f53/extractor/0.log" Jul 13 22:19:42.868692 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:42.868658 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-nrbc6" Jul 13 22:19:43.884661 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:43.884581 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-ptxzb_f41427fd-dc84-4aec-aeb7-2a70a840f081/manager/0.log" Jul 13 22:19:44.069339 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:44.069304 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hjj6p_a108cde0-a23b-46fc-81a7-85d22163d74e/manager/0.log" Jul 13 22:19:44.188004 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:44.187974 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9swdn_cd2ea4b2-0027-4cd7-8e29-e36db1367a36/server/0.log" Jul 13 22:19:44.281047 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:44.281014 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-57hdq_7d3a8120-670f-41a4-bfcb-1cce715a150b/manager/0.log" Jul 13 22:19:44.433698 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:44.433667 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-728pc_06e0de3a-b207-4d77-b51c-2e05e01b3b5c/manager/0.log" Jul 13 22:19:44.454256 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:44.454181 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-728pc_06e0de3a-b207-4d77-b51c-2e05e01b3b5c/copy-manifests/0.log" Jul 13 22:19:45.843959 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:45.843919 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-h9qz2_66dc8bfa-4f8b-4a83-ba22-7fdf8aa5b8f6/openshift-lws-operator/0.log" Jul 13 22:19:46.309521 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:46.309496 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4npv5l_95af6daa-68a4-4861-95d7-40c4ee1c761a/istio-proxy/0.log" Jul 13 22:19:46.455667 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:46.455631 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-kh9fs_3d654a0c-921f-475d-b656-2795bd12952a/kube-auth-proxy/0.log" Jul 13 22:19:46.477233 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:46.477200 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/0.log" Jul 13 22:19:46.520402 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:46.520362 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/1.log" Jul 13 22:19:47.663478 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:47.663418 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4npv5l_95af6daa-68a4-4861-95d7-40c4ee1c761a/istio-proxy/0.log" Jul 13 22:19:47.814699 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:47.814633 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-kh9fs_3d654a0c-921f-475d-b656-2795bd12952a/kube-auth-proxy/0.log" Jul 13 22:19:47.834728 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:47.834701 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/0.log" Jul 13 22:19:47.876384 ip-10-0-133-19 kubenswrapper[2583]: I0713 22:19:47.876360 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tfxd7_cc3b4a1a-acbb-4915-9e03-94f5ccfa3a4b/istio-proxy/1.log"