Jul 20 16:14:11.687875 ip-10-0-131-211 systemd[1]: Starting Kubernetes Kubelet... Jul 20 16:14:12.077111 ip-10-0-131-211 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:14:12.077111 ip-10-0-131-211 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 16:14:12.077111 ip-10-0-131-211 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:14:12.077111 ip-10-0-131-211 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 16:14:12.077684 ip-10-0-131-211 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:14:12.079367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.079279 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 16:14:12.083098 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083084 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:14:12.083098 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083098 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083101 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083105 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083107 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083110 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083113 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083115 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083118 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083120 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083123 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083125 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083128 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083130 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083133 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083136 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083139 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083142 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083144 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083147 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:14:12.083153 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083150 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083153 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083156 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083158 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083161 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083164 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083167 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083170 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083172 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083174 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083177 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083181 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083185 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083189 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083192 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083194 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083197 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083199 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083201 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083204 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:14:12.083595 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083207 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083209 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083212 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083215 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083217 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083220 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083222 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083224 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083227 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083229 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083233 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083236 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083239 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083242 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083246 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083249 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083253 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083256 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083258 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:14:12.084070 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083261 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083264 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083267 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083269 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083272 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083274 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083277 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083280 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083282 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083285 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083288 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083291 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083293 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083296 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083299 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083301 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083305 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083309 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083312 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083315 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:14:12.084511 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083317 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083320 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083323 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083325 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083327 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083330 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083332 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083335 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083338 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083341 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083343 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083346 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083348 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083351 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083353 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083356 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083358 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083360 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083363 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.083365 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:14:12.085016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084745 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084753 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084758 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084761 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084764 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084767 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084769 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084772 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084774 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084777 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084780 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084782 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084785 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084787 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084789 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084792 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084795 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084797 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084800 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:14:12.085477 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084805 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084808 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084810 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084813 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084816 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084818 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084821 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084823 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084827 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084831 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084833 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084836 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084839 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084841 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084844 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084846 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084848 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084855 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084858 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084860 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:14:12.085950 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084863 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084865 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084868 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084870 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084873 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084875 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084879 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084881 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084884 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084886 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084889 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084893 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084897 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084900 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084902 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084905 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084908 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084910 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084913 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084915 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:14:12.086404 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084917 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084920 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084922 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084925 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084927 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084929 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084932 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084934 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084936 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084939 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084947 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084950 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084952 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084954 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084957 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084961 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084963 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084966 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084968 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:14:12.086893 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084970 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084973 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084975 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084978 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084980 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084984 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084987 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084989 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084992 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084994 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084997 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.084999 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085003 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085006 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085009 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085011 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085014 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085016 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085019 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085021 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:14:12.087332 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085024 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085393 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085399 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085403 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085405 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085408 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085411 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085414 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085417 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085419 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085422 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085424 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085427 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085430 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085432 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085435 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085438 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085441 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085443 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085446 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:14:12.087816 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085449 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085451 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085454 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085456 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085459 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085462 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085464 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085467 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085469 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085471 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085474 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085476 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085479 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085482 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085484 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085487 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085490 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085493 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085495 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085497 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:14:12.088284 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085500 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085502 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085505 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085507 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085510 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085513 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085516 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085518 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085522 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085524 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085527 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085529 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085532 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085534 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085536 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085539 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085541 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085544 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085547 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085549 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:14:12.088760 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085552 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085555 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085557 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085560 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085563 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085567 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085569 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085572 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085575 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085578 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085580 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085583 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085585 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085587 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085590 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085592 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085594 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085597 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085599 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:14:12.089229 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085602 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085604 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085607 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085611 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085627 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085630 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085633 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085635 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085638 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085640 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085643 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085645 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085648 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085650 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085653 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085655 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085658 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085660 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085664 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:14:12.089730 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085667 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.085670 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085730 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085736 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085742 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085746 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085750 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085753 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085757 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085761 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085764 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085768 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085771 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085774 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085777 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085781 2582 flags.go:64] FLAG: --cgroup-root="" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085784 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085787 2582 flags.go:64] FLAG: --client-ca-file="" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085790 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085792 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085796 2582 flags.go:64] FLAG: --cluster-domain="" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085799 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085802 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085805 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 16:14:12.090179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085808 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085812 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085815 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085818 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085821 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085824 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085826 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085829 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085832 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085835 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085839 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085842 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085845 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085848 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085851 2582 flags.go:64] FLAG: --enable-server="true" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085854 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085860 2582 flags.go:64] FLAG: --event-burst="100" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085863 2582 flags.go:64] FLAG: --event-qps="50" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085865 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085868 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085871 2582 flags.go:64] FLAG: --eviction-hard="" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085875 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085877 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085880 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085883 2582 flags.go:64] FLAG: --eviction-soft="" Jul 20 16:14:12.090785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085886 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085889 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085892 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085895 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085898 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085901 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085903 2582 flags.go:64] FLAG: --feature-gates="" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085907 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085909 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085912 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085915 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085918 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085921 2582 flags.go:64] FLAG: --help="false" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085924 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085927 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085929 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085932 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085936 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085939 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085942 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085944 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085948 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085951 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 16:14:12.091369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085954 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085957 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085959 2582 flags.go:64] FLAG: --kube-reserved="" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085963 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085966 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085969 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085974 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085977 2582 flags.go:64] FLAG: --lock-file="" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085979 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085983 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085986 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085991 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085994 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085997 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.085999 2582 flags.go:64] FLAG: --logging-format="text" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086002 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086005 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086008 2582 flags.go:64] FLAG: --manifest-url="" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086011 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086015 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086018 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086021 2582 flags.go:64] FLAG: --max-pods="110" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086024 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086027 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086030 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 16:14:12.092035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086033 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086035 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086038 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086041 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086048 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086055 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086059 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086062 2582 flags.go:64] FLAG: --pod-cidr="" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086065 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086070 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086073 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086076 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086079 2582 flags.go:64] FLAG: --port="10250" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086083 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086086 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e01664016b454984" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086089 2582 flags.go:64] FLAG: --qos-reserved="" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086092 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086095 2582 flags.go:64] FLAG: --register-node="true" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086097 2582 flags.go:64] FLAG: --register-with-taints="" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086101 2582 flags.go:64] FLAG: --registry-burst="10" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086104 2582 flags.go:64] FLAG: --registry-qps="5" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086106 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086109 2582 flags.go:64] FLAG: --reserved-memory="" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086113 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086116 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 16:14:12.092614 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086119 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086122 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086124 2582 flags.go:64] FLAG: --runonce="false" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086127 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086130 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086133 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086136 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086138 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086141 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086144 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086147 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086150 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086152 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086155 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086163 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086167 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086169 2582 flags.go:64] FLAG: --system-cgroups="" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086172 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086177 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086180 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086184 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086188 2582 flags.go:64] FLAG: --tls-min-version="" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086190 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086193 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086196 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 16:14:12.093263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086199 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086202 2582 flags.go:64] FLAG: --v="2" Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086205 2582 flags.go:64] FLAG: --version="false" Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086216 2582 flags.go:64] FLAG: --vmodule="" Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086220 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.086223 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086334 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086337 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086340 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086342 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086344 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086347 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086350 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086352 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086355 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086359 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086362 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086364 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086366 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086369 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086372 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:14:12.093876 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086374 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086382 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086385 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086388 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086390 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086393 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086398 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086400 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086404 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086408 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086410 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086413 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086415 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086418 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086421 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086424 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086426 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086429 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086433 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:14:12.094652 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086436 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086439 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086442 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086444 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086447 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086449 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086452 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086455 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086458 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086461 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086463 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086466 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086468 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086471 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086473 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086481 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086484 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086487 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086489 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086493 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:14:12.095207 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086496 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086498 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086501 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086503 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086506 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086508 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086511 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086513 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086516 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086518 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086520 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086523 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086525 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086528 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086530 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086533 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086535 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086537 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086540 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086543 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:14:12.095713 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086546 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086548 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086551 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086553 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086556 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086558 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086561 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086563 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086571 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086574 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086576 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086580 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086582 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086585 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086587 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086590 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086592 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086595 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086597 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086601 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:14:12.096180 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086604 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086607 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086610 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086612 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.086628 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.087202 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.093340 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.093353 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.093380 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.093386 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093436 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093442 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093445 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093449 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093451 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:14:12.096664 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093454 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093457 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093459 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093462 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093465 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093467 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093470 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093473 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093475 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093478 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093480 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093482 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093485 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093488 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093490 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093493 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093495 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093498 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093500 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:14:12.097016 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093502 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093505 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093507 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093510 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093513 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093517 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093519 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093522 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093524 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093527 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093529 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093532 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093534 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093537 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093539 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093541 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093544 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093546 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093549 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093551 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:14:12.097461 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093554 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093556 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093558 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093561 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093564 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093566 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093570 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093573 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093576 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093579 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093581 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093584 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093586 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093589 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093591 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093594 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093596 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093599 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093603 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:14:12.097963 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093607 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093610 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093613 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093632 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093636 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093638 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093641 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093643 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093646 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093648 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093651 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093653 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093656 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093658 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093660 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093663 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093665 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093668 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093670 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093672 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:14:12.098398 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093675 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093677 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093680 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093682 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093684 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093687 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093689 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093691 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093694 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093696 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093699 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093701 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093705 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093708 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093710 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:14:12.098904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093712 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.093717 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093916 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093924 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093927 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093930 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093934 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093941 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093944 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093947 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093949 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093952 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093955 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093957 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093960 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:14:12.099255 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.093963 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094577 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094594 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094605 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094613 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094634 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094639 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094644 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094649 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094654 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094659 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094663 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094674 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094679 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094683 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094687 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094692 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094696 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094700 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:14:12.099704 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094704 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094708 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094712 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094716 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094721 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094725 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094734 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094738 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094742 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094746 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094750 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094754 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094758 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094762 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094766 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094772 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094776 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094780 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094785 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094794 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:14:12.100143 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094798 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094803 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094807 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094811 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094815 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094889 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094897 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094903 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094908 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094913 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094918 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094923 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094928 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094933 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094938 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094962 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094965 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094968 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094971 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.094974 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:14:12.100609 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095075 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095081 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095084 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095086 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095089 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095091 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095094 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095097 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095100 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095103 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095105 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095108 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095110 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095115 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095119 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095121 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095124 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095127 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095129 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:14:12.101093 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095137 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095140 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095143 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095146 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095149 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095151 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095154 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:12.095157 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.095162 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.095260 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.097244 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.098687 2582 server.go:1019] "Starting client certificate rotation" Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.098784 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.098812 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 16:14:12.101553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.101269 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 16:14:12.102361 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.102344 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 16:14:12.111239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.111224 2582 log.go:25] "Validated CRI v1 runtime API" Jul 20 16:14:12.115186 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.115172 2582 log.go:25] "Validated CRI v1 image API" Jul 20 16:14:12.115960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.115946 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 16:14:12.118995 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.118977 2582 fs.go:135] Filesystem UUIDs: map[10dd9ad3-5bcd-46a0-b4f8-c6b072635690:/dev/nvme0n1p3 4e2ff8f1-e0c2-415d-9663-0fe75fab60ea:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 20 16:14:12.119051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.118995 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 16:14:12.124014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.123901 2582 manager.go:216] Machine: {Timestamp:2026-07-20 16:14:12.122651734 +0000 UTC m=+0.327797466 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199727 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27105a2da63c28b92a951f0ea382a3 SystemUUID:ec27105a-2da6-3c28-b92a-951f0ea382a3 BootID:25c96d4f-4b8e-4058-bfeb-9e8c2a552a93 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ae:90:8f:24:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ae:90:8f:24:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:7d:43:7e:92:96 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 16:14:12.124014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.124008 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 16:14:12.124170 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.124099 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 16:14:12.126298 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.126275 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 16:14:12.126413 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.126300 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-211.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 16:14:12.126782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.126768 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 16:14:12.126936 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.126926 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 16:14:12.126970 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.126939 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 16:14:12.126970 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.126945 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 16:14:12.127539 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.127530 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 16:14:12.128752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.128742 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 16:14:12.131361 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131350 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 20 16:14:12.131464 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131455 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 16:14:12.131500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131483 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 20 16:14:12.131500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131492 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 16:14:12.131560 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131505 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 16:14:12.131560 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131514 2582 kubelet.go:398] "Adding apiserver pod source" Jul 20 16:14:12.131560 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131522 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 16:14:12.131815 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131804 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 16:14:12.131859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.131826 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 16:14:12.136808 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.136788 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 16:14:12.138632 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.138580 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 16:14:12.138632 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.138630 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 16:14:12.139868 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139855 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 16:14:12.139868 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139871 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139877 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139882 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139890 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139895 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139906 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139912 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139918 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139924 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139932 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 16:14:12.139948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.139937 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 16:14:12.141135 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.141114 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-211.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 16:14:12.141135 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.141115 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 16:14:12.141312 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.141301 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 16:14:12.141312 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.141313 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 16:14:12.144123 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.144110 2582 server.go:1268] "Started kubelet" Jul 20 16:14:12.144231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.144204 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 16:14:12.144307 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.144257 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 16:14:12.144400 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.144322 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 16:14:12.145668 ip-10-0-131-211 systemd[1]: Started Kubernetes Kubelet. Jul 20 16:14:12.145978 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.145957 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 16:14:12.146065 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.146023 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 20 16:14:12.148804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.148788 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-211.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 16:14:12.149594 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.149564 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 16:14:12.149758 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.148779 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-211.ec2.internal.18c40b8ce6fad7a7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-211.ec2.internal,UID:ip-10-0-131-211.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-211.ec2.internal,},FirstTimestamp:2026-07-20 16:14:12.144084903 +0000 UTC m=+0.349230637,LastTimestamp:2026-07-20 16:14:12.144084903 +0000 UTC m=+0.349230637,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-211.ec2.internal,}" Jul 20 16:14:12.151651 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.151638 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 16:14:12.151651 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.151644 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 16:14:12.151765 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.151708 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.152261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152237 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 16:14:12.152261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152240 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 16:14:12.152381 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152272 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 16:14:12.152381 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152351 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 20 16:14:12.152381 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152360 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 20 16:14:12.152381 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152380 2582 factory.go:55] Registering systemd factory Jul 20 16:14:12.152545 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152394 2582 factory.go:223] Registration of the systemd container factory successfully Jul 20 16:14:12.152631 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152604 2582 factory.go:153] Registering CRI-O factory Jul 20 16:14:12.152679 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152634 2582 factory.go:223] Registration of the crio container factory successfully Jul 20 16:14:12.152770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152690 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 16:14:12.152770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152708 2582 factory.go:103] Registering Raw factory Jul 20 16:14:12.152770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.152724 2582 manager.go:1196] Started watching for new ooms in manager Jul 20 16:14:12.153031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.153019 2582 manager.go:349] Starting recovery of all containers Jul 20 16:14:12.156368 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.156344 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 16:14:12.156447 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.156403 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-211.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 16:14:12.158908 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.158884 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 16:14:12.161915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.161891 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gq9nk" Jul 20 16:14:12.163313 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.163294 2582 manager.go:354] Recovery completed Jul 20 16:14:12.168642 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.168613 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gq9nk" Jul 20 16:14:12.168708 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.168696 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.170801 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.170788 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.170862 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.170814 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.170862 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.170826 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.171308 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.171296 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 16:14:12.171308 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.171308 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 16:14:12.171387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.171308 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.171387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.171325 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 20 16:14:12.171984 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.171966 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.172064 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.171993 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.172064 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.172003 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.172426 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.172414 2582 policy_none.go:49] "None policy: Start" Jul 20 16:14:12.172465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.172431 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 16:14:12.172465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.172441 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 16:14:12.172867 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.172791 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-211.ec2.internal.18c40b8ce8927d08 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-211.ec2.internal,UID:ip-10-0-131-211.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-211.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-211.ec2.internal,},FirstTimestamp:2026-07-20 16:14:12.170800392 +0000 UTC m=+0.375946126,LastTimestamp:2026-07-20 16:14:12.170800392 +0000 UTC m=+0.375946126,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-211.ec2.internal,}" Jul 20 16:14:12.173435 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.173425 2582 policy_none.go:47] "Start" Jul 20 16:14:12.212242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212132 2582 manager.go:337] "Starting Device Plugin manager" Jul 20 16:14:12.212325 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.212261 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 16:14:12.212325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212274 2582 server.go:87] "Starting device plugin registration server" Jul 20 16:14:12.212485 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212469 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 16:14:12.212523 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212482 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 16:14:12.212564 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212555 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 16:14:12.212663 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212647 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 16:14:12.212663 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.212660 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 16:14:12.213180 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.213153 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 16:14:12.213247 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.213222 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.313006 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.312976 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.314259 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.313737 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.314259 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.313761 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.314259 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.313773 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.314259 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.313795 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.319460 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.319442 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.319460 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.319459 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-211.ec2.internal\": node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.321383 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.321358 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 16:14:12.321448 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.321394 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 16:14:12.321448 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.321413 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 16:14:12.321448 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.321439 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 16:14:12.323221 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.323202 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:14:12.334467 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.334423 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.422588 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.422548 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal"] Jul 20 16:14:12.422686 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.422636 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.423386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.423371 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.423453 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.423395 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.423453 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.423405 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.424747 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.424734 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.424885 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.424870 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.424979 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.424906 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.425372 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.425357 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.425443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.425362 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.425443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.425412 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.425443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.425424 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.425443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.425387 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.425579 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.425463 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.426620 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.426605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.426685 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.426628 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:14:12.427297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.427283 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:14:12.427380 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.427307 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:14:12.427380 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.427321 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:14:12.435572 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.435554 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.452382 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.452361 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-211.ec2.internal\" not found" node="ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.453042 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.453026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ad93a0f02d30ff499207190d61ffb0a2-config\") pod \"kube-apiserver-proxy-ip-10-0-131-211.ec2.internal\" (UID: \"ad93a0f02d30ff499207190d61ffb0a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.453110 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.453054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac2e99c6d0aec45a76b0abe4820ee3fd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal\" (UID: \"ac2e99c6d0aec45a76b0abe4820ee3fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.453110 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.453078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e99c6d0aec45a76b0abe4820ee3fd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal\" (UID: \"ac2e99c6d0aec45a76b0abe4820ee3fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.456863 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.456850 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-211.ec2.internal\" not found" node="ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.536398 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.536373 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.553928 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.553910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ad93a0f02d30ff499207190d61ffb0a2-config\") pod \"kube-apiserver-proxy-ip-10-0-131-211.ec2.internal\" (UID: \"ad93a0f02d30ff499207190d61ffb0a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.554000 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.553953 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac2e99c6d0aec45a76b0abe4820ee3fd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal\" (UID: \"ac2e99c6d0aec45a76b0abe4820ee3fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.554000 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.553986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ad93a0f02d30ff499207190d61ffb0a2-config\") pod \"kube-apiserver-proxy-ip-10-0-131-211.ec2.internal\" (UID: \"ad93a0f02d30ff499207190d61ffb0a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.554109 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.554002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac2e99c6d0aec45a76b0abe4820ee3fd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal\" (UID: \"ac2e99c6d0aec45a76b0abe4820ee3fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.554109 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.554045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e99c6d0aec45a76b0abe4820ee3fd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal\" (UID: \"ac2e99c6d0aec45a76b0abe4820ee3fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.554109 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.554081 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e99c6d0aec45a76b0abe4820ee3fd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal\" (UID: \"ac2e99c6d0aec45a76b0abe4820ee3fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.637324 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.637275 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.741873 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.741845 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.755135 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.755116 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.759864 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:12.759846 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" Jul 20 16:14:12.842220 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.842189 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:12.942923 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:12.942872 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:13.043407 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:13.043376 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:13.051282 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.051264 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:14:13.098389 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.098370 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 16:14:13.098758 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.098468 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:14:13.098758 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.098466 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:14:13.144294 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:13.144268 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-211.ec2.internal\" not found" Jul 20 16:14:13.147266 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.147249 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:14:13.152230 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.152215 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" Jul 20 16:14:13.152303 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.152235 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 16:14:13.164272 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.164251 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 16:14:13.165373 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.165361 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" Jul 20 16:14:13.166329 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.166314 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:14:13.166440 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.166422 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 16:14:13.171563 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.171549 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 16:14:13.172723 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.172694 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 16:09:12 +0000 UTC" deadline="2028-04-19 11:18:28.136674086 +0000 UTC" Jul 20 16:14:13.172778 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.172726 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15331h4m14.96395208s" Jul 20 16:14:13.190215 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.190195 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nr47g" Jul 20 16:14:13.198174 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.198130 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nr47g" Jul 20 16:14:13.440635 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:13.440596 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad93a0f02d30ff499207190d61ffb0a2.slice/crio-7c87317af1c0978e0318b976250227d14cb4fd45d645c958d060e01d51bbc611 WatchSource:0}: Error finding container 7c87317af1c0978e0318b976250227d14cb4fd45d645c958d060e01d51bbc611: Status 404 returned error can't find the container with id 7c87317af1c0978e0318b976250227d14cb4fd45d645c958d060e01d51bbc611 Jul 20 16:14:13.446826 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:13.446809 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:14:13.462863 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:13.462841 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2e99c6d0aec45a76b0abe4820ee3fd.slice/crio-9fb6d56b3a8560092509cd9ba19898a5f60a77035d78d86bb91c11d5ac16b0fe WatchSource:0}: Error finding container 9fb6d56b3a8560092509cd9ba19898a5f60a77035d78d86bb91c11d5ac16b0fe: Status 404 returned error can't find the container with id 9fb6d56b3a8560092509cd9ba19898a5f60a77035d78d86bb91c11d5ac16b0fe Jul 20 16:14:14.133584 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.133552 2582 apiserver.go:52] "Watching apiserver" Jul 20 16:14:14.143724 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.143684 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 16:14:14.145743 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.145703 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-rlr9z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76","openshift-image-registry/node-ca-szh9h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal","openshift-multus/multus-additional-cni-plugins-4x6m8","openshift-multus/multus-qwjv7","openshift-multus/network-metrics-daemon-6vk8k","openshift-network-operator/iptables-alerter-xb89p","kube-system/global-pull-secret-syncer-szff5","kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal","openshift-cluster-node-tuning-operator/tuned-4g4fs","openshift-dns/node-resolver-hlwlt","openshift-network-diagnostics/network-check-target-xswbf","openshift-ovn-kubernetes/ovnkube-node-jxd8x"] Jul 20 16:14:14.147546 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.147522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:14.147680 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.147640 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:14.149590 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.149568 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.151095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.151070 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.152116 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.152094 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gcpg5\"" Jul 20 16:14:14.152970 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.152950 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.153183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.153167 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.153302 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.153289 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 16:14:14.153500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.153485 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.153904 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.153882 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8vx62\"" Jul 20 16:14:14.154098 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.154073 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 16:14:14.154320 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.154162 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.154630 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.154468 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.155496 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.156770 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.156990 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pgxsl\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.157248 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.157494 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.158014 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.158269 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bckdt\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.158486 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.158737 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.158982 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.159194 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.159373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.159589 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.159439 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:14.161067 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.160390 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2dkxn\"" Jul 20 16:14:14.161067 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.160417 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.161067 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.160642 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 16:14:14.161471 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.161332 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.161471 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.161399 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:14.161828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.161809 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-device-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.161993 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.161974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtdw\" (UniqueName: \"kubernetes.io/projected/91c83092-5ab9-46c4-aa1f-78091ac02328-kube-api-access-tjtdw\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.162191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.162174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5mvj\" (UniqueName: \"kubernetes.io/projected/081af134-9948-49cf-9b25-a05824539331-kube-api-access-b5mvj\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.162366 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.162350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3081a2dc-7efc-4631-ab62-cc41fa045f67-agent-certs\") pod \"konnectivity-agent-rlr9z\" (UID: \"3081a2dc-7efc-4631-ab62-cc41fa045f67\") " pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.162613 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.162595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.164031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.162976 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.164031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.163083 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 16:14:14.164031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.163128 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hbp9x\"" Jul 20 16:14:14.164031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.163380 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.163459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-socket-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-registration-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164162 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/081af134-9948-49cf-9b25-a05824539331-host\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164184 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/081af134-9948-49cf-9b25-a05824539331-serviceca\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164207 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3081a2dc-7efc-4631-ab62-cc41fa045f67-konnectivity-ca\") pod \"konnectivity-agent-rlr9z\" (UID: \"3081a2dc-7efc-4631-ab62-cc41fa045f67\") " pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:14.164279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164257 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-etc-selinux\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.164625 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.164283 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-sys-fs\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.165515 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.165493 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.166800 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.166780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.168782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.168782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168215 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 16:14:14.168782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168307 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.168782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168550 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 16:14:14.168782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168652 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.169089 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168861 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 16:14:14.169089 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.168956 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.169089 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.169032 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8spmc\"" Jul 20 16:14:14.169363 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.169319 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 16:14:14.169808 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.169635 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.169808 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.169676 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g6wfj\"" Jul 20 16:14:14.170813 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.170763 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 16:14:14.170998 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.170976 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-48jxw\"" Jul 20 16:14:14.171500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.171482 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 16:14:14.200088 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.199983 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:09:13 +0000 UTC" deadline="2028-02-03 11:14:35.406331394 +0000 UTC" Jul 20 16:14:14.200088 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.200034 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13507h0m21.206301508s" Jul 20 16:14:14.253235 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.253210 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 16:14:14.264931 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.264902 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysctl-conf\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.265039 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.264945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-socket-dir-parent\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265039 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.264970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-dbus\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.265039 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.264993 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-slash\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265039 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265020 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-etc-selinux\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-kubernetes\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265069 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a404e52-cd74-46b2-9e3b-610090ae6c97-cni-binary-copy\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265086 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-kubelet-config\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-etc-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-kubelet\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-device-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265148 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-tuned\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-registration-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-cnibin\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265228 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv62m\" (UniqueName: \"kubernetes.io/projected/6a404e52-cd74-46b2-9e3b-610090ae6c97-kube-api-access-qv62m\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-systemd-units\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-cni-bin\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265279 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-systemd\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265330 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265347 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-daemon-config\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-etc-kubernetes\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovnkube-config\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-env-overrides\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265476 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265485 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psl8f\" (UniqueName: \"kubernetes.io/projected/d451b4ed-751d-4b7e-84ef-9b5b90265b96-kube-api-access-psl8f\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265505 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysctl-d\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-registration-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265615 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21384c1f-0989-4be9-ad26-4586bf5e1511-tmp-dir\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265633 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-hostroot\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.265895 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265650 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265668 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-run-netns\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-run-ovn-kubernetes\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5mvj\" (UniqueName: \"kubernetes.io/projected/081af134-9948-49cf-9b25-a05824539331-kube-api-access-b5mvj\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3081a2dc-7efc-4631-ab62-cc41fa045f67-agent-certs\") pod \"konnectivity-agent-rlr9z\" (UID: \"3081a2dc-7efc-4631-ab62-cc41fa045f67\") " pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-run\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265800 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-os-release\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265827 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x46zq\" (UniqueName: \"kubernetes.io/projected/d89302d9-1985-400e-a259-c9fcd79c0e77-kube-api-access-x46zq\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lg5b\" (UniqueName: \"kubernetes.io/projected/b42b9ca6-2705-49d0-9b6f-90490e762b1c-kube-api-access-6lg5b\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3081a2dc-7efc-4631-ab62-cc41fa045f67-konnectivity-ca\") pod \"konnectivity-agent-rlr9z\" (UID: \"3081a2dc-7efc-4631-ab62-cc41fa045f67\") " pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-cni-binary-copy\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265865 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-device-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.265969 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-etc-selinux\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b42b9ca6-2705-49d0-9b6f-90490e762b1c-iptables-alerter-script\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-kubelet\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.266658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266076 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-ovn\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-node-log\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266142 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-sys-fs\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266168 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-conf-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266202 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-log-socket\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266236 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266759 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3081a2dc-7efc-4631-ab62-cc41fa045f67-konnectivity-ca\") pod \"konnectivity-agent-rlr9z\" (UID: \"3081a2dc-7efc-4631-ab62-cc41fa045f67\") " pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266877 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-sys-fs\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-system-cni-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-os-release\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.266982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-host\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267012 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21384c1f-0989-4be9-ad26-4586bf5e1511-hosts-file\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267034 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-system-cni-dir\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267065 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-cni-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-cni-bin\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b42b9ca6-2705-49d0-9b6f-90490e762b1c-host-slash\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-var-lib-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.267500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267260 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-lib-modules\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-socket-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/081af134-9948-49cf-9b25-a05824539331-host\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/081af134-9948-49cf-9b25-a05824539331-serviceca\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267447 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/081af134-9948-49cf-9b25-a05824539331-host\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bc895fe-a283-4a9a-8157-24130e0fe838-tmp\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-multus-certs\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267576 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovnkube-script-lib\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267602 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-modprobe-d\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267626 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-sys\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267690 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-cnibin\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-netns\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-systemd\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267807 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysconfig\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267832 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-var-lib-kubelet\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.268350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267864 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j9p8\" (UniqueName: \"kubernetes.io/projected/7bc895fe-a283-4a9a-8157-24130e0fe838-kube-api-access-8j9p8\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.267890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nrg7\" (UniqueName: \"kubernetes.io/projected/21384c1f-0989-4be9-ad26-4586bf5e1511-kube-api-access-8nrg7\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-k8s-cni-cncf-io\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268052 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/081af134-9948-49cf-9b25-a05824539331-serviceca\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-cni-multus\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8lqv\" (UniqueName: \"kubernetes.io/projected/f70698c0-e1fe-42e1-a111-cacbd6efcddd-kube-api-access-s8lqv\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268165 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtdw\" (UniqueName: \"kubernetes.io/projected/91c83092-5ab9-46c4-aa1f-78091ac02328-kube-api-access-tjtdw\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268172 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91c83092-5ab9-46c4-aa1f-78091ac02328-socket-dir\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-cni-netd\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.269193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.268222 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovn-node-metrics-cert\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.270591 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.270567 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3081a2dc-7efc-4631-ab62-cc41fa045f67-agent-certs\") pod \"konnectivity-agent-rlr9z\" (UID: \"3081a2dc-7efc-4631-ab62-cc41fa045f67\") " pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.271983 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.271961 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:14.272082 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.271989 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:14.272082 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.272004 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:14.272195 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.272107 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:14:14.772085332 +0000 UTC m=+2.977231073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:14.275346 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.275286 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5mvj\" (UniqueName: \"kubernetes.io/projected/081af134-9948-49cf-9b25-a05824539331-kube-api-access-b5mvj\") pod \"node-ca-szh9h\" (UID: \"081af134-9948-49cf-9b25-a05824539331\") " pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.276856 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.276830 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtdw\" (UniqueName: \"kubernetes.io/projected/91c83092-5ab9-46c4-aa1f-78091ac02328-kube-api-access-tjtdw\") pod \"aws-ebs-csi-driver-node-mrx76\" (UID: \"91c83092-5ab9-46c4-aa1f-78091ac02328\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.326477 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.326430 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" event={"ID":"ac2e99c6d0aec45a76b0abe4820ee3fd","Type":"ContainerStarted","Data":"9fb6d56b3a8560092509cd9ba19898a5f60a77035d78d86bb91c11d5ac16b0fe"} Jul 20 16:14:14.329362 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.328658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" event={"ID":"ad93a0f02d30ff499207190d61ffb0a2","Type":"ContainerStarted","Data":"7c87317af1c0978e0318b976250227d14cb4fd45d645c958d060e01d51bbc611"} Jul 20 16:14:14.368530 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368503 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-systemd\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.368688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.368688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-daemon-config\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.368688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-etc-kubernetes\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.368688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368617 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovnkube-config\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.368688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-env-overrides\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.368961 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368671 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psl8f\" (UniqueName: \"kubernetes.io/projected/d451b4ed-751d-4b7e-84ef-9b5b90265b96-kube-api-access-psl8f\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.368961 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368750 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-etc-kubernetes\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.368961 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.368849 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-systemd\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.369175 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369137 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysctl-d\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.369239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369192 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21384c1f-0989-4be9-ad26-4586bf5e1511-tmp-dir\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.369239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-hostroot\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.369342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.369342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369247 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-daemon-config\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.369342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369269 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-run-netns\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.369342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369294 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-run-ovn-kubernetes\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.369542 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-run\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.369542 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369369 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-os-release\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.369542 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369392 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovnkube-config\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.369542 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369404 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.369542 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369456 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21384c1f-0989-4be9-ad26-4586bf5e1511-tmp-dir\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.369542 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.369491 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.369553 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:14:14.869535596 +0000 UTC m=+3.074681334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369572 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysctl-d\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369492 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-hostroot\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369611 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-run-netns\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x46zq\" (UniqueName: \"kubernetes.io/projected/d89302d9-1985-400e-a259-c9fcd79c0e77-kube-api-access-x46zq\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369668 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-run-ovn-kubernetes\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369670 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lg5b\" (UniqueName: \"kubernetes.io/projected/b42b9ca6-2705-49d0-9b6f-90490e762b1c-kube-api-access-6lg5b\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-os-release\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369800 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-cni-binary-copy\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b42b9ca6-2705-49d0-9b6f-90490e762b1c-iptables-alerter-script\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.369874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369864 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-kubelet\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-ovn\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-node-log\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-conf-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-run\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369982 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-log-socket\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370034 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-kubelet\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.369945 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-node-log\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-system-cni-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-os-release\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-host\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370122 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-system-cni-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370384 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-cni-binary-copy\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.370476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-env-overrides\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370741 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b42b9ca6-2705-49d0-9b6f-90490e762b1c-iptables-alerter-script\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370747 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-conf-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370758 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-log-socket\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21384c1f-0989-4be9-ad26-4586bf5e1511-hosts-file\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370816 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-system-cni-dir\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370872 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-host\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-cni-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370869 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-ovn\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370914 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-system-cni-dir\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-cni-bin\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370969 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21384c1f-0989-4be9-ad26-4586bf5e1511-hosts-file\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b42b9ca6-2705-49d0-9b6f-90490e762b1c-host-slash\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.370976 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-cni-bin\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371000 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-cni-dir\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371024 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-var-lib-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-os-release\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-lib-modules\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.371188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371087 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b42b9ca6-2705-49d0-9b6f-90490e762b1c-host-slash\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bc895fe-a283-4a9a-8157-24130e0fe838-tmp\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371090 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-var-lib-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371165 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-lib-modules\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371202 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-multus-certs\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371265 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovnkube-script-lib\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-modprobe-d\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371306 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-multus-certs\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-sys\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-cnibin\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-netns\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371436 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-sys\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-systemd\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysconfig\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-var-lib-kubelet\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371521 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j9p8\" (UniqueName: \"kubernetes.io/projected/7bc895fe-a283-4a9a-8157-24130e0fe838-kube-api-access-8j9p8\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372095 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371526 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-modprobe-d\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371551 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nrg7\" (UniqueName: \"kubernetes.io/projected/21384c1f-0989-4be9-ad26-4586bf5e1511-kube-api-access-8nrg7\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371576 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-k8s-cni-cncf-io\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-cni-multus\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371606 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysconfig\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.371625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8lqv\" (UniqueName: \"kubernetes.io/projected/f70698c0-e1fe-42e1-a111-cacbd6efcddd-kube-api-access-s8lqv\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372654 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d89302d9-1985-400e-a259-c9fcd79c0e77-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-cni-netd\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372730 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-k8s-cni-cncf-io\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372790 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovn-node-metrics-cert\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovnkube-script-lib\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysctl-conf\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.372929 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372945 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-socket-dir-parent\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.372975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-dbus\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.372994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373006 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-slash\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.373023 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:14:14.873006324 +0000 UTC m=+3.078152070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373066 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-slash\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-kubernetes\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373114 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a404e52-cd74-46b2-9e3b-610090ae6c97-cni-binary-copy\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373144 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-kubelet-config\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373170 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-etc-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-sysctl-conf\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373239 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-etc-openvswitch\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373276 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-multus-socket-dir-parent\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373411 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-dbus\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-run-systemd\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373651 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-cnibin\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.373840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373788 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-run-netns\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373874 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-cni-netd\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-cni-multus\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.373956 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-var-lib-kubelet\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374242 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-kubelet-config\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374277 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a404e52-cd74-46b2-9e3b-610090ae6c97-cni-binary-copy\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-kubelet\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.374461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-tuned\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374476 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bc895fe-a283-4a9a-8157-24130e0fe838-tmp\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-cnibin\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a404e52-cd74-46b2-9e3b-610090ae6c97-host-var-lib-kubelet\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374588 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv62m\" (UniqueName: \"kubernetes.io/projected/6a404e52-cd74-46b2-9e3b-610090ae6c97-kube-api-access-qv62m\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374619 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d89302d9-1985-400e-a259-c9fcd79c0e77-cnibin\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374624 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-systemd-units\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374692 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-systemd-units\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.374840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374831 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-cni-bin\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.375234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.375234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374940 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-cni-bin\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.375234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.374962 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d451b4ed-751d-4b7e-84ef-9b5b90265b96-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.376335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.376310 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d451b4ed-751d-4b7e-84ef-9b5b90265b96-ovn-node-metrics-cert\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.377409 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.377390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-kubernetes\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.377790 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.377767 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7bc895fe-a283-4a9a-8157-24130e0fe838-etc-tuned\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.385416 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.385107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lg5b\" (UniqueName: \"kubernetes.io/projected/b42b9ca6-2705-49d0-9b6f-90490e762b1c-kube-api-access-6lg5b\") pod \"iptables-alerter-xb89p\" (UID: \"b42b9ca6-2705-49d0-9b6f-90490e762b1c\") " pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.387753 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.387728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nrg7\" (UniqueName: \"kubernetes.io/projected/21384c1f-0989-4be9-ad26-4586bf5e1511-kube-api-access-8nrg7\") pod \"node-resolver-hlwlt\" (UID: \"21384c1f-0989-4be9-ad26-4586bf5e1511\") " pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.388200 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.388175 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psl8f\" (UniqueName: \"kubernetes.io/projected/d451b4ed-751d-4b7e-84ef-9b5b90265b96-kube-api-access-psl8f\") pod \"ovnkube-node-jxd8x\" (UID: \"d451b4ed-751d-4b7e-84ef-9b5b90265b96\") " pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.388680 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.388652 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8lqv\" (UniqueName: \"kubernetes.io/projected/f70698c0-e1fe-42e1-a111-cacbd6efcddd-kube-api-access-s8lqv\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.389136 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.389114 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j9p8\" (UniqueName: \"kubernetes.io/projected/7bc895fe-a283-4a9a-8157-24130e0fe838-kube-api-access-8j9p8\") pod \"tuned-4g4fs\" (UID: \"7bc895fe-a283-4a9a-8157-24130e0fe838\") " pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.389562 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.389539 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x46zq\" (UniqueName: \"kubernetes.io/projected/d89302d9-1985-400e-a259-c9fcd79c0e77-kube-api-access-x46zq\") pod \"multus-additional-cni-plugins-4x6m8\" (UID: \"d89302d9-1985-400e-a259-c9fcd79c0e77\") " pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.391787 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.391751 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv62m\" (UniqueName: \"kubernetes.io/projected/6a404e52-cd74-46b2-9e3b-610090ae6c97-kube-api-access-qv62m\") pod \"multus-qwjv7\" (UID: \"6a404e52-cd74-46b2-9e3b-610090ae6c97\") " pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.404807 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.404780 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:14:14.470325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.470296 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" Jul 20 16:14:14.477981 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.477960 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c83092_5ab9_46c4_aa1f_78091ac02328.slice/crio-d6830c735d598efb0ced58590791bff6cf34bd8e9e96df2a730fd1c643cc2624 WatchSource:0}: Error finding container d6830c735d598efb0ced58590791bff6cf34bd8e9e96df2a730fd1c643cc2624: Status 404 returned error can't find the container with id d6830c735d598efb0ced58590791bff6cf34bd8e9e96df2a730fd1c643cc2624 Jul 20 16:14:14.479134 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.479119 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-szh9h" Jul 20 16:14:14.486345 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.486320 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081af134_9948_49cf_9b25_a05824539331.slice/crio-7ad07c0559a37fbedafb549c15e40698bf33fae2dc7ddc2142a686186adfd951 WatchSource:0}: Error finding container 7ad07c0559a37fbedafb549c15e40698bf33fae2dc7ddc2142a686186adfd951: Status 404 returned error can't find the container with id 7ad07c0559a37fbedafb549c15e40698bf33fae2dc7ddc2142a686186adfd951 Jul 20 16:14:14.489443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.489410 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:14.497579 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.497552 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3081a2dc_7efc_4631_ab62_cc41fa045f67.slice/crio-c99a5d80d8c341f2c34dd57b705fd4a3cbb60ad0d4b91742d9f805522a65bca1 WatchSource:0}: Error finding container c99a5d80d8c341f2c34dd57b705fd4a3cbb60ad0d4b91742d9f805522a65bca1: Status 404 returned error can't find the container with id c99a5d80d8c341f2c34dd57b705fd4a3cbb60ad0d4b91742d9f805522a65bca1 Jul 20 16:14:14.502388 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.502368 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" Jul 20 16:14:14.510608 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.510588 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qwjv7" Jul 20 16:14:14.510870 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.510848 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89302d9_1985_400e_a259_c9fcd79c0e77.slice/crio-02fd401a91d2041f6fdcc9174360eea0fa2a2fef75dba668d3e95ee287938618 WatchSource:0}: Error finding container 02fd401a91d2041f6fdcc9174360eea0fa2a2fef75dba668d3e95ee287938618: Status 404 returned error can't find the container with id 02fd401a91d2041f6fdcc9174360eea0fa2a2fef75dba668d3e95ee287938618 Jul 20 16:14:14.519156 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.519131 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a404e52_cd74_46b2_9e3b_610090ae6c97.slice/crio-7e9bc29ab2d9fc076c002a08507c7789305cdb01a3140ba38dcab842168ff5b4 WatchSource:0}: Error finding container 7e9bc29ab2d9fc076c002a08507c7789305cdb01a3140ba38dcab842168ff5b4: Status 404 returned error can't find the container with id 7e9bc29ab2d9fc076c002a08507c7789305cdb01a3140ba38dcab842168ff5b4 Jul 20 16:14:14.519937 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.519918 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xb89p" Jul 20 16:14:14.528398 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.528376 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:14.528943 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.528912 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42b9ca6_2705_49d0_9b6f_90490e762b1c.slice/crio-b7001b7a86894f0eeeaa48166aef74ad76369d81fbc6ca7e1d23381a49ed69a7 WatchSource:0}: Error finding container b7001b7a86894f0eeeaa48166aef74ad76369d81fbc6ca7e1d23381a49ed69a7: Status 404 returned error can't find the container with id b7001b7a86894f0eeeaa48166aef74ad76369d81fbc6ca7e1d23381a49ed69a7 Jul 20 16:14:14.536906 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.536888 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" Jul 20 16:14:14.537006 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:14.536987 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd451b4ed_751d_4b7e_84ef_9b5b90265b96.slice/crio-6dfe102858d3faa76018b63149d845005152d90d19f5c2538249465cb7cc82ea WatchSource:0}: Error finding container 6dfe102858d3faa76018b63149d845005152d90d19f5c2538249465cb7cc82ea: Status 404 returned error can't find the container with id 6dfe102858d3faa76018b63149d845005152d90d19f5c2538249465cb7cc82ea Jul 20 16:14:14.545561 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.545505 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hlwlt" Jul 20 16:14:14.778598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.778521 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:14.778765 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.778683 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:14.778765 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.778706 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:14.778765 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.778718 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:14.778935 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.778783 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:14:15.778761382 +0000 UTC m=+3.983907103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:14.879387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.879353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:14.879521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:14.879431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:14.879521 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.879487 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:14.879602 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.879538 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:14.879602 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.879549 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:14:15.879533411 +0000 UTC m=+4.084679132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:14.879602 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:14.879593 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:14:15.87957595 +0000 UTC m=+4.084721690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:15.200425 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.200332 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:09:13 +0000 UTC" deadline="2028-02-23 23:50:02.501838741 +0000 UTC" Jul 20 16:14:15.200425 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.200381 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13999h35m47.301460701s" Jul 20 16:14:15.205947 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:15.205917 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc895fe_a283_4a9a_8157_24130e0fe838.slice/crio-ac2574afcf37dc500300fc2b2b9a14a1638fb8abf7855a400bd3736c0d6ed613 WatchSource:0}: Error finding container ac2574afcf37dc500300fc2b2b9a14a1638fb8abf7855a400bd3736c0d6ed613: Status 404 returned error can't find the container with id ac2574afcf37dc500300fc2b2b9a14a1638fb8abf7855a400bd3736c0d6ed613 Jul 20 16:14:15.321980 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.321903 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:15.322132 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.322026 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:15.332239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.332161 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" event={"ID":"7bc895fe-a283-4a9a-8157-24130e0fe838","Type":"ContainerStarted","Data":"ac2574afcf37dc500300fc2b2b9a14a1638fb8abf7855a400bd3736c0d6ed613"} Jul 20 16:14:15.333714 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.333683 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"6dfe102858d3faa76018b63149d845005152d90d19f5c2538249465cb7cc82ea"} Jul 20 16:14:15.335298 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.335253 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xb89p" event={"ID":"b42b9ca6-2705-49d0-9b6f-90490e762b1c","Type":"ContainerStarted","Data":"b7001b7a86894f0eeeaa48166aef74ad76369d81fbc6ca7e1d23381a49ed69a7"} Jul 20 16:14:15.336881 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.336851 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerStarted","Data":"02fd401a91d2041f6fdcc9174360eea0fa2a2fef75dba668d3e95ee287938618"} Jul 20 16:14:15.338811 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.338733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-szh9h" event={"ID":"081af134-9948-49cf-9b25-a05824539331","Type":"ContainerStarted","Data":"7ad07c0559a37fbedafb549c15e40698bf33fae2dc7ddc2142a686186adfd951"} Jul 20 16:14:15.340336 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.340288 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" event={"ID":"91c83092-5ab9-46c4-aa1f-78091ac02328","Type":"ContainerStarted","Data":"d6830c735d598efb0ced58590791bff6cf34bd8e9e96df2a730fd1c643cc2624"} Jul 20 16:14:15.341885 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.341861 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qwjv7" event={"ID":"6a404e52-cd74-46b2-9e3b-610090ae6c97","Type":"ContainerStarted","Data":"7e9bc29ab2d9fc076c002a08507c7789305cdb01a3140ba38dcab842168ff5b4"} Jul 20 16:14:15.343411 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.343358 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rlr9z" event={"ID":"3081a2dc-7efc-4631-ab62-cc41fa045f67","Type":"ContainerStarted","Data":"c99a5d80d8c341f2c34dd57b705fd4a3cbb60ad0d4b91742d9f805522a65bca1"} Jul 20 16:14:15.785698 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.785552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:15.785842 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.785785 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:15.785950 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.785897 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:15.785950 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.785942 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:15.786065 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.786003 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:14:17.785985692 +0000 UTC m=+5.991131430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:15.886767 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.886736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:15.886876 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:15.886808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:15.886974 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.886956 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:15.887044 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.887031 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:14:17.887010131 +0000 UTC m=+6.092155851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:15.887436 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.887417 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:15.887516 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:15.887472 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:14:17.887457098 +0000 UTC m=+6.092602838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:16.158685 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:16.158565 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:14:16.326060 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:16.325737 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:16.326060 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:16.325961 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:16.326539 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:16.326165 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:16.326539 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:16.326285 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:16.363012 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:16.362932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hlwlt" event={"ID":"21384c1f-0989-4be9-ad26-4586bf5e1511","Type":"ContainerStarted","Data":"9501680cd996e8caa39052fc936db49077e87bdf7ba628588ac7d1f564be0fa6"} Jul 20 16:14:16.372914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:16.372199 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" event={"ID":"ad93a0f02d30ff499207190d61ffb0a2","Type":"ContainerStarted","Data":"8e8a24eaee29727dc239c0231a64327a513015f7ee4e52274be058ef0e9b33a6"} Jul 20 16:14:16.387214 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:16.387163 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-211.ec2.internal" podStartSLOduration=3.387146937 podStartE2EDuration="3.387146937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:16.386440163 +0000 UTC m=+4.591585905" watchObservedRunningTime="2026-07-20 16:14:16.387146937 +0000 UTC m=+4.592292680" Jul 20 16:14:17.322117 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:17.322081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:17.322285 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.322220 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:17.384274 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:17.383253 2582 generic.go:356] "Generic (PLEG): container finished" podID="ac2e99c6d0aec45a76b0abe4820ee3fd" containerID="3bcadc2a8942ddbc3700b2280eb347b8d3ad33c4a8d950c7aef3fdab70bebd6c" exitCode=0 Jul 20 16:14:17.384274 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:17.384221 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" event={"ID":"ac2e99c6d0aec45a76b0abe4820ee3fd","Type":"ContainerDied","Data":"3bcadc2a8942ddbc3700b2280eb347b8d3ad33c4a8d950c7aef3fdab70bebd6c"} Jul 20 16:14:17.803869 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:17.803790 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:17.804045 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.803983 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:17.804045 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.804042 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:17.804159 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.804055 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:17.804159 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.804119 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:14:21.804099901 +0000 UTC m=+10.009245638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:17.904885 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:17.904854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:17.905035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:17.904937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:17.905181 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.905162 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:17.905249 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.905229 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:14:21.9052092 +0000 UTC m=+10.110354938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:17.905693 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.905673 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:17.905797 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:17.905726 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:14:21.905711701 +0000 UTC m=+10.110857439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:18.323911 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:18.323403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:18.323911 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:18.323420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:18.323911 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:18.323538 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:18.323911 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:18.323585 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:19.322687 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:19.322155 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:19.322687 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:19.322279 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:20.321871 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:20.321836 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:20.322049 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:20.321879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:20.322049 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:20.321984 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:20.322161 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:20.322052 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:21.322087 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:21.322048 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:21.322526 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.322185 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:21.838036 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:21.838004 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:21.838208 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.838188 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:21.838335 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.838213 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:21.838335 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.838223 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:21.838335 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.838273 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:14:29.838258964 +0000 UTC m=+18.043404683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:21.939350 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:21.939288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:21.939528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:21.939357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:21.939528 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.939471 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:21.939528 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.939473 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:21.939695 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.939534 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:14:29.939514789 +0000 UTC m=+18.144660522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:21.939695 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:21.939552 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:14:29.939542293 +0000 UTC m=+18.144688035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:22.323285 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:22.323194 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:22.323744 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:22.323307 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:22.323744 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:22.323705 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:22.323851 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:22.323830 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:23.322348 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:23.322318 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:23.322516 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:23.322451 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:24.322062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:24.321577 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:24.322062 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:24.321716 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:24.322062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:24.321577 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:24.322062 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:24.321890 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:25.321594 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:25.321566 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:25.321725 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:25.321704 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:26.322528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:26.322493 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:26.322991 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:26.322599 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:26.322991 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:26.322722 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:26.322991 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:26.322837 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:27.321573 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:27.321543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:27.321754 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:27.321668 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:28.322081 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:28.322043 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:28.322598 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:28.322194 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:28.322598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:28.322208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:28.322598 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:28.322300 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:29.321845 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:29.321769 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:29.322022 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.321905 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:29.895146 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:29.895112 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:29.895503 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.895265 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:29.895503 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.895288 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:29.895503 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.895302 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:29.895503 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.895360 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:14:45.895345048 +0000 UTC m=+34.100490783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:29.996135 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:29.996102 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:29.996333 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:29.996157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:29.996333 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.996254 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:29.996333 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.996310 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:14:45.996295524 +0000 UTC m=+34.201441248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:29.996333 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.996254 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:29.996537 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:29.996354 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:14:45.996343867 +0000 UTC m=+34.201489588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:30.322636 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:30.322547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:30.322790 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:30.322686 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:30.322790 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:30.322751 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:30.322922 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:30.322897 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:31.322186 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:31.322151 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:31.322635 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:31.322264 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:32.323585 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:32.323548 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:32.324020 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:32.323690 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:32.324020 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:32.323740 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:32.324020 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:32.323850 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:33.322326 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.322304 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:33.322662 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:33.322406 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:33.415958 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.415751 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rlr9z" event={"ID":"3081a2dc-7efc-4631-ab62-cc41fa045f67","Type":"ContainerStarted","Data":"2372ee68c54dedfcd7fc84326a19f4d4d68900afb20a0f504afc6dfb883c32c1"} Jul 20 16:14:33.417497 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.417460 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerStarted","Data":"8ea389db72cf1da2362ddbb1937622bfb84ccdb5fdec5df9e7f6bc597d935119"} Jul 20 16:14:33.419104 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.419079 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-szh9h" event={"ID":"081af134-9948-49cf-9b25-a05824539331","Type":"ContainerStarted","Data":"c2d00c02a834ca6ede4e6da21916be508743402bfcf2da7a3d4a3f01a5d4cd82"} Jul 20 16:14:33.420583 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.420537 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" event={"ID":"91c83092-5ab9-46c4-aa1f-78091ac02328","Type":"ContainerStarted","Data":"156451f4dd579f0f15f2ddd51da65b5310e6b14b310c5474fc60010d7660a3f5"} Jul 20 16:14:33.422014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.421993 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hlwlt" event={"ID":"21384c1f-0989-4be9-ad26-4586bf5e1511","Type":"ContainerStarted","Data":"ae18d84fde2a77b6f8b8d6ea1e0c081ecd57e49b9647934963605ac34f5e2f6f"} Jul 20 16:14:33.425802 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.425595 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" event={"ID":"ac2e99c6d0aec45a76b0abe4820ee3fd","Type":"ContainerStarted","Data":"1408583e002b4ab6362e270d8e70f188525c57a6a6b0f66eafe6ea2c193b243b"} Jul 20 16:14:33.454437 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.454390 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rlr9z" podStartSLOduration=6.966613421 podStartE2EDuration="21.454373274s" totalImagesPullingTime="14.487759853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:33.43072986 +0000 UTC m=+21.635875602" watchObservedRunningTime="2026-07-20 16:14:33.454373274 +0000 UTC m=+21.659519012" Jul 20 16:14:33.467331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.467287 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-211.ec2.internal" podStartSLOduration=20.467274762 podStartE2EDuration="20.467274762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:33.466946989 +0000 UTC m=+21.672092731" watchObservedRunningTime="2026-07-20 16:14:33.467274762 +0000 UTC m=+21.672420502" Jul 20 16:14:33.486177 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.486129 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hlwlt" podStartSLOduration=4.224197868 podStartE2EDuration="21.486112922s" totalImagesPullingTime="17.261915054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:33.485855659 +0000 UTC m=+21.691001400" watchObservedRunningTime="2026-07-20 16:14:33.486112922 +0000 UTC m=+21.691258662" Jul 20 16:14:33.501283 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:33.501241 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-szh9h" podStartSLOduration=7.00166645 podStartE2EDuration="21.501230872s" totalImagesPullingTime="14.499564422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:33.501157335 +0000 UTC m=+21.706303076" watchObservedRunningTime="2026-07-20 16:14:33.501230872 +0000 UTC m=+21.706376610" Jul 20 16:14:34.322484 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.322380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:34.322571 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.322434 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:34.322635 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:34.322592 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:34.322693 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:34.322676 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:34.348663 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.348642 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-szh9h_081af134-9948-49cf-9b25-a05824539331/node-ca/0.log" Jul 20 16:14:34.428160 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.428135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" event={"ID":"7bc895fe-a283-4a9a-8157-24130e0fe838","Type":"ContainerStarted","Data":"a4e029eddbcf52f1e58c4a8541f32a0528d5d27b7179beaf8ca31efd6922afb0"} Jul 20 16:14:34.429724 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.429707 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"c699d1dd967f347c4e37064130d676724cb67e8e23ee80a9e844acc34c6efdff"} Jul 20 16:14:34.429804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.429769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"cec81891de759506233c1724115c9e6e2c6b65f2935242c823d61311e3157e0f"} Jul 20 16:14:34.430903 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.430882 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xb89p" event={"ID":"b42b9ca6-2705-49d0-9b6f-90490e762b1c","Type":"ContainerStarted","Data":"b7be2e96e8fd787fce5ce3f1cead04b1eff26a24e6d4311800a5129eef613b0f"} Jul 20 16:14:34.432140 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.432120 2582 generic.go:356] "Generic (PLEG): container finished" podID="d89302d9-1985-400e-a259-c9fcd79c0e77" containerID="8ea389db72cf1da2362ddbb1937622bfb84ccdb5fdec5df9e7f6bc597d935119" exitCode=0 Jul 20 16:14:34.432224 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.432205 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerDied","Data":"8ea389db72cf1da2362ddbb1937622bfb84ccdb5fdec5df9e7f6bc597d935119"} Jul 20 16:14:34.434959 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.434939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qwjv7" event={"ID":"6a404e52-cd74-46b2-9e3b-610090ae6c97","Type":"ContainerStarted","Data":"09eedd2483d7066b9c6f277cb2ec3cb2435daeca652e2c55a3f893434bd42b5f"} Jul 20 16:14:34.446847 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.446805 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4g4fs" podStartSLOduration=4.54864357 podStartE2EDuration="22.446794515s" totalImagesPullingTime="17.898150945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:34.446329218 +0000 UTC m=+22.651474959" watchObservedRunningTime="2026-07-20 16:14:34.446794515 +0000 UTC m=+22.651940298" Jul 20 16:14:34.487264 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.487232 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qwjv7" podStartSLOduration=3.754897007 podStartE2EDuration="22.487220177s" totalImagesPullingTime="18.73232317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:34.462907053 +0000 UTC m=+22.668052793" watchObservedRunningTime="2026-07-20 16:14:34.487220177 +0000 UTC m=+22.692365917" Jul 20 16:14:34.565280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:34.565262 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 16:14:35.226801 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.226648 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T16:14:34.565275083Z","UUID":"eb2e793d-6768-43fb-ae6d-2c729a1bffad","Handler":null,"Name":"","Endpoint":""} Jul 20 16:14:35.228520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.228501 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 16:14:35.228664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.228551 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 16:14:35.322467 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.322442 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:35.322637 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:35.322532 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:35.439010 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.438979 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" event={"ID":"91c83092-5ab9-46c4-aa1f-78091ac02328","Type":"ContainerStarted","Data":"dd1f7ec156d39d980d29cb5d3a3136e5545ef5b64616d361a1ab32f2cdb7ffd3"} Jul 20 16:14:35.444263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.444224 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"2a8ae7cb22e5d74c1f6dc73948b7ce9d9d7b3d724c01a3f2c398ea576c0373f3"} Jul 20 16:14:35.444398 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.444306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"5bcac985c71d48719fa5805ad1f7d8790d2140c934a420d31be195a61e156023"} Jul 20 16:14:35.444398 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.444353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"5d0a854b579f1d6dfd57d16fc429516cf755eed9a8345c5386687a1831cbdadd"} Jul 20 16:14:35.444493 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:35.444399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"35ae0eb7beac98a1a26b43d76705ffb0e837bc030c74dee14a1d7b96ea56f904"} Jul 20 16:14:36.321924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:36.321702 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:36.322124 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:36.321716 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:36.322124 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:36.322030 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:36.322124 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:36.322094 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:36.448119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:36.448090 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" event={"ID":"91c83092-5ab9-46c4-aa1f-78091ac02328","Type":"ContainerStarted","Data":"550b6da1d620c156f16eb520ccec0d201dad1ca4d7f6b83c641498fdde5ed461"} Jul 20 16:14:36.465962 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:36.465916 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xb89p" podStartSLOduration=5.893248655 podStartE2EDuration="24.465897955s" totalImagesPullingTime="18.5726493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:34.502412558 +0000 UTC m=+22.707558299" watchObservedRunningTime="2026-07-20 16:14:36.465897955 +0000 UTC m=+24.671043700" Jul 20 16:14:36.466516 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:36.466482 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrx76" podStartSLOduration=3.53279519 podStartE2EDuration="24.466468241s" totalImagesPullingTime="20.933673051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:36.465549706 +0000 UTC m=+24.670695447" watchObservedRunningTime="2026-07-20 16:14:36.466468241 +0000 UTC m=+24.671613985" Jul 20 16:14:37.206897 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:37.206824 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:37.207449 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:37.207418 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:37.321773 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:37.321748 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:37.321890 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:37.321840 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:37.453105 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:37.453074 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"469d0341b96e35c7f4f9ff6372de59cae1b8b4346ef283e9d0e60c9f906a51fc"} Jul 20 16:14:37.453645 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:37.453283 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:37.453756 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:37.453741 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rlr9z" Jul 20 16:14:38.321856 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:38.321826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:38.322011 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:38.321924 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:38.322075 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:38.322008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:38.324642 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:38.322456 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:39.322030 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:39.321861 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:39.322460 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:39.322110 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:39.461101 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:39.460984 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" event={"ID":"d451b4ed-751d-4b7e-84ef-9b5b90265b96","Type":"ContainerStarted","Data":"45debcde8357256f5021a7eb663194f0f19aafc6ed7e765576484116215d327b"} Jul 20 16:14:39.488906 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:39.488863 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" podStartSLOduration=8.030662853 podStartE2EDuration="27.488847297s" totalImagesPullingTime="19.458184444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:39.488440114 +0000 UTC m=+27.693585855" watchObservedRunningTime="2026-07-20 16:14:39.488847297 +0000 UTC m=+27.693993040" Jul 20 16:14:40.321831 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.321802 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:40.321831 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.321817 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:40.322055 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:40.321916 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:40.322370 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:40.322067 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:40.464031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.463943 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:40.464031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.464031 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:40.464219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.464078 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:40.481405 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.481378 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:40.481629 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:40.481597 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:14:41.321822 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:41.321773 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:41.321906 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:41.321864 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:41.467545 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:41.467518 2582 generic.go:356] "Generic (PLEG): container finished" podID="d89302d9-1985-400e-a259-c9fcd79c0e77" containerID="a86c2d512daef01eb272ef7b293f8ffb7c3532e1180f04ecd2d21900fd410252" exitCode=0 Jul 20 16:14:41.467862 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:41.467609 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerDied","Data":"a86c2d512daef01eb272ef7b293f8ffb7c3532e1180f04ecd2d21900fd410252"} Jul 20 16:14:42.322536 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:42.322514 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:42.322658 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:42.322593 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:42.322714 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:42.322692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:42.322814 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:42.322787 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:42.470682 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:42.470605 2582 generic.go:356] "Generic (PLEG): container finished" podID="d89302d9-1985-400e-a259-c9fcd79c0e77" containerID="ed42c8b9eb7e7d163bc8260a1a5c65f796bf0093d038ef33b1b211771a81cead" exitCode=0 Jul 20 16:14:42.470977 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:42.470690 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerDied","Data":"ed42c8b9eb7e7d163bc8260a1a5c65f796bf0093d038ef33b1b211771a81cead"} Jul 20 16:14:43.322003 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:43.321927 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:43.322132 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:43.322031 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:43.474289 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:43.474256 2582 generic.go:356] "Generic (PLEG): container finished" podID="d89302d9-1985-400e-a259-c9fcd79c0e77" containerID="e1fc7a66beb261e1500383df5a2a0270116197d3a49a0b2113521c0845362758" exitCode=0 Jul 20 16:14:43.474598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:43.474305 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerDied","Data":"e1fc7a66beb261e1500383df5a2a0270116197d3a49a0b2113521c0845362758"} Jul 20 16:14:44.322137 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:44.322106 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:44.322332 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:44.322214 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:44.322332 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:44.322255 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:44.322332 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:44.322309 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:45.322159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:45.322127 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:45.322562 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:45.322248 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:45.919015 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:45.918981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:45.919182 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:45.919159 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:14:45.919182 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:45.919181 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:14:45.919294 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:45.919194 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5xnz for pod openshift-network-diagnostics/network-check-target-xswbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:45.919294 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:45.919255 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz podName:2b52bcb1-7c46-4c77-a949-d5d261f243ad nodeName:}" failed. No retries permitted until 2026-07-20 16:15:17.91923591 +0000 UTC m=+66.124381645 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5xnz" (UniqueName: "kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz") pod "network-check-target-xswbf" (UID: "2b52bcb1-7c46-4c77-a949-d5d261f243ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:14:46.019835 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:46.019629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:46.019971 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:46.019881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:46.019971 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:46.019711 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:46.019971 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:46.019970 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret podName:6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4 nodeName:}" failed. No retries permitted until 2026-07-20 16:15:18.019948249 +0000 UTC m=+66.225093972 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret") pod "global-pull-secret-syncer-szff5" (UID: "6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:14:46.020107 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:46.019971 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:46.020107 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:46.020020 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs podName:f70698c0-e1fe-42e1-a111-cacbd6efcddd nodeName:}" failed. No retries permitted until 2026-07-20 16:15:18.020004449 +0000 UTC m=+66.225150174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs") pod "network-metrics-daemon-6vk8k" (UID: "f70698c0-e1fe-42e1-a111-cacbd6efcddd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:14:46.321897 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:46.321824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:46.321897 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:46.321836 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:46.322192 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:46.321952 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:46.322192 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:46.322061 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:47.322297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:47.322259 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:47.322728 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:47.322388 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:48.322011 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:48.321988 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:48.322154 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:48.322134 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:48.322219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:48.322179 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:48.322281 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:48.322265 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:48.489077 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:48.489024 2582 generic.go:356] "Generic (PLEG): container finished" podID="d89302d9-1985-400e-a259-c9fcd79c0e77" containerID="71bf8eb75d25b40dd059df7fe284c992ba2b3ce2c8426e8102bf98f250acf0b5" exitCode=0 Jul 20 16:14:48.489347 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:48.489079 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerDied","Data":"71bf8eb75d25b40dd059df7fe284c992ba2b3ce2c8426e8102bf98f250acf0b5"} Jul 20 16:14:49.321606 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:49.321577 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:49.321768 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:49.321674 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:49.493802 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:49.493768 2582 generic.go:356] "Generic (PLEG): container finished" podID="d89302d9-1985-400e-a259-c9fcd79c0e77" containerID="ee913b501ab2da2c443ebf8547ad07919301740c7bc5380b015d0d1f78a0ee90" exitCode=0 Jul 20 16:14:49.494137 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:49.493818 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerDied","Data":"ee913b501ab2da2c443ebf8547ad07919301740c7bc5380b015d0d1f78a0ee90"} Jul 20 16:14:50.321557 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:50.321532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:50.321722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:50.321583 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:50.321722 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:50.321664 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:50.321816 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:50.321788 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:50.498329 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:50.498300 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" event={"ID":"d89302d9-1985-400e-a259-c9fcd79c0e77","Type":"ContainerStarted","Data":"c6524f63acc22c94986d66f6d2cba7d131bd69680c0f570ca170641b69c7f687"} Jul 20 16:14:50.521601 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:50.521560 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4x6m8" podStartSLOduration=7.347276168 podStartE2EDuration="38.521546472s" totalImagesPullingTime="31.174270304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:14:50.521106919 +0000 UTC m=+38.726252663" watchObservedRunningTime="2026-07-20 16:14:50.521546472 +0000 UTC m=+38.726692212" Jul 20 16:14:51.321938 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:51.321912 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:51.322101 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:51.322007 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:52.324280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:52.324246 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:52.324682 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:52.324370 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:52.324682 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:52.324416 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:52.324682 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:52.324500 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:53.322244 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:53.322211 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:53.322390 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:53.322303 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:54.321996 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:54.321965 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:54.322439 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:54.322089 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:54.322439 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:54.322160 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:54.322439 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:54.322245 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:55.321983 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.321961 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:55.322079 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:55.322060 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:55.459599 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.459572 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6vk8k"] Jul 20 16:14:55.459761 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.459717 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:55.459853 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:55.459832 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:55.462163 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.462135 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-szff5"] Jul 20 16:14:55.462294 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.462235 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:55.462350 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:55.462324 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:55.462835 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.462815 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xswbf"] Jul 20 16:14:55.511517 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:55.511461 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:55.511603 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:55.511548 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:57.321789 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:57.321762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:57.321789 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:57.321792 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:57.322144 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:57.321867 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szff5" podUID="6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4" Jul 20 16:14:57.322144 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:57.321905 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6vk8k" podUID="f70698c0-e1fe-42e1-a111-cacbd6efcddd" Jul 20 16:14:57.322144 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:57.321932 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:57.322144 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:14:57.321973 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xswbf" podUID="2b52bcb1-7c46-4c77-a949-d5d261f243ad" Jul 20 16:14:59.181470 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.181304 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-211.ec2.internal" event="NodeReady" Jul 20 16:14:59.181851 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.181563 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 16:14:59.226069 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.226041 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dkt8q"] Jul 20 16:14:59.260660 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.260638 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ff5cq"] Jul 20 16:14:59.260797 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.260774 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.263215 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.263194 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 16:14:59.263320 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.263201 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 16:14:59.263399 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.263388 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6tgrd\"" Jul 20 16:14:59.274466 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.274449 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ff5cq"] Jul 20 16:14:59.274585 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.274480 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkt8q"] Jul 20 16:14:59.274585 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.274506 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4bdxf"] Jul 20 16:14:59.274585 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.274558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.277314 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.277300 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 16:14:59.277395 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.277363 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 16:14:59.277503 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.277490 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 16:14:59.277644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.277629 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sqlzb\"" Jul 20 16:14:59.296327 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.296305 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4bdxf"] Jul 20 16:14:59.296419 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.296405 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.299504 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.299488 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 16:14:59.299802 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.299780 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 16:14:59.299878 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.299866 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lzr47\"" Jul 20 16:14:59.299936 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.299882 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 16:14:59.299984 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.299960 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 16:14:59.322442 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.322427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:14:59.322525 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.322432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:14:59.322572 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.322531 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:14:59.325651 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.325633 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 16:14:59.325744 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.325718 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 16:14:59.325803 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.325742 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 16:14:59.325803 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.325785 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fbfhg\"" Jul 20 16:14:59.325900 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.325861 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sngxb\"" Jul 20 16:14:59.325953 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.325915 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 16:14:59.416421 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416399 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e6ee43-be51-4ce7-9617-25dafef1add4-config-volume\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.416521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416426 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/a4851e3b-efcd-4291-8c66-61efff445cc6-kube-api-access-nmzkg\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.416521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416445 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r65x7\" (UniqueName: \"kubernetes.io/projected/b6fde27d-e433-493b-ab11-b73ccf046951-kube-api-access-r65x7\") pod \"ingress-canary-ff5cq\" (UID: \"b6fde27d-e433-493b-ab11-b73ccf046951\") " pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.416521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4e6ee43-be51-4ce7-9617-25dafef1add4-metrics-tls\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.416521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416477 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4851e3b-efcd-4291-8c66-61efff445cc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.416521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6fde27d-e433-493b-ab11-b73ccf046951-cert\") pod \"ingress-canary-ff5cq\" (UID: \"b6fde27d-e433-493b-ab11-b73ccf046951\") " pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.416521 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416522 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e6ee43-be51-4ce7-9617-25dafef1add4-tmp-dir\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.416752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416540 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4851e3b-efcd-4291-8c66-61efff445cc6-data-volume\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.416752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4851e3b-efcd-4291-8c66-61efff445cc6-crio-socket\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.416752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416659 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjrpl\" (UniqueName: \"kubernetes.io/projected/c4e6ee43-be51-4ce7-9617-25dafef1add4-kube-api-access-kjrpl\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.416752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.416680 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4851e3b-efcd-4291-8c66-61efff445cc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.517573 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e6ee43-be51-4ce7-9617-25dafef1add4-config-volume\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.517573 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/a4851e3b-efcd-4291-8c66-61efff445cc6-kube-api-access-nmzkg\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.517743 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517578 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r65x7\" (UniqueName: \"kubernetes.io/projected/b6fde27d-e433-493b-ab11-b73ccf046951-kube-api-access-r65x7\") pod \"ingress-canary-ff5cq\" (UID: \"b6fde27d-e433-493b-ab11-b73ccf046951\") " pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.517743 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4e6ee43-be51-4ce7-9617-25dafef1add4-metrics-tls\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.517743 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4851e3b-efcd-4291-8c66-61efff445cc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.517743 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6fde27d-e433-493b-ab11-b73ccf046951-cert\") pod \"ingress-canary-ff5cq\" (UID: \"b6fde27d-e433-493b-ab11-b73ccf046951\") " pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.517918 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517766 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e6ee43-be51-4ce7-9617-25dafef1add4-tmp-dir\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.517918 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4851e3b-efcd-4291-8c66-61efff445cc6-data-volume\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.517918 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517851 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4851e3b-efcd-4291-8c66-61efff445cc6-crio-socket\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.517918 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjrpl\" (UniqueName: \"kubernetes.io/projected/c4e6ee43-be51-4ce7-9617-25dafef1add4-kube-api-access-kjrpl\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.517918 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.517907 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4851e3b-efcd-4291-8c66-61efff445cc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.518449 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.518196 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e6ee43-be51-4ce7-9617-25dafef1add4-config-volume\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.518449 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.518308 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4851e3b-efcd-4291-8c66-61efff445cc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.518449 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.518330 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4851e3b-efcd-4291-8c66-61efff445cc6-crio-socket\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.518655 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.518586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4851e3b-efcd-4291-8c66-61efff445cc6-data-volume\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.518788 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.518771 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e6ee43-be51-4ce7-9617-25dafef1add4-tmp-dir\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.521819 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.521802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4e6ee43-be51-4ce7-9617-25dafef1add4-metrics-tls\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.521898 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.521838 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4851e3b-efcd-4291-8c66-61efff445cc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.521898 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.521856 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6fde27d-e433-493b-ab11-b73ccf046951-cert\") pod \"ingress-canary-ff5cq\" (UID: \"b6fde27d-e433-493b-ab11-b73ccf046951\") " pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.525454 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.525436 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r65x7\" (UniqueName: \"kubernetes.io/projected/b6fde27d-e433-493b-ab11-b73ccf046951-kube-api-access-r65x7\") pod \"ingress-canary-ff5cq\" (UID: \"b6fde27d-e433-493b-ab11-b73ccf046951\") " pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.525595 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.525562 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/a4851e3b-efcd-4291-8c66-61efff445cc6-kube-api-access-nmzkg\") pod \"insights-runtime-extractor-4bdxf\" (UID: \"a4851e3b-efcd-4291-8c66-61efff445cc6\") " pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.525758 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.525738 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjrpl\" (UniqueName: \"kubernetes.io/projected/c4e6ee43-be51-4ce7-9617-25dafef1add4-kube-api-access-kjrpl\") pod \"dns-default-dkt8q\" (UID: \"c4e6ee43-be51-4ce7-9617-25dafef1add4\") " pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.570195 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.570176 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkt8q" Jul 20 16:14:59.582284 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.582267 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ff5cq" Jul 20 16:14:59.603922 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.603907 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4bdxf" Jul 20 16:14:59.775051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.774975 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ff5cq"] Jul 20 16:14:59.775769 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.775744 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkt8q"] Jul 20 16:14:59.781642 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:59.780541 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e6ee43_be51_4ce7_9617_25dafef1add4.slice/crio-962656a0821df261981ec3a46dc7c3ba83f78013156f50417dcc408215dcea6c WatchSource:0}: Error finding container 962656a0821df261981ec3a46dc7c3ba83f78013156f50417dcc408215dcea6c: Status 404 returned error can't find the container with id 962656a0821df261981ec3a46dc7c3ba83f78013156f50417dcc408215dcea6c Jul 20 16:14:59.782182 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:59.782159 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6fde27d_e433_493b_ab11_b73ccf046951.slice/crio-057768d2f9e4d4b4824e8d96370a7c2f68623a9ccb6cd67fbcaaf7c730a7c81e WatchSource:0}: Error finding container 057768d2f9e4d4b4824e8d96370a7c2f68623a9ccb6cd67fbcaaf7c730a7c81e: Status 404 returned error can't find the container with id 057768d2f9e4d4b4824e8d96370a7c2f68623a9ccb6cd67fbcaaf7c730a7c81e Jul 20 16:14:59.796455 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:14:59.796436 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4bdxf"] Jul 20 16:14:59.797966 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:14:59.797948 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4851e3b_efcd_4291_8c66_61efff445cc6.slice/crio-244a4d926c8a20d2adf94e677b0f3c01c3b0ac90696689f0621e7ca0a08f54bb WatchSource:0}: Error finding container 244a4d926c8a20d2adf94e677b0f3c01c3b0ac90696689f0621e7ca0a08f54bb: Status 404 returned error can't find the container with id 244a4d926c8a20d2adf94e677b0f3c01c3b0ac90696689f0621e7ca0a08f54bb Jul 20 16:15:00.527299 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:00.525037 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bdxf" event={"ID":"a4851e3b-efcd-4291-8c66-61efff445cc6","Type":"ContainerStarted","Data":"244a4d926c8a20d2adf94e677b0f3c01c3b0ac90696689f0621e7ca0a08f54bb"} Jul 20 16:15:00.527299 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:00.526349 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ff5cq" event={"ID":"b6fde27d-e433-493b-ab11-b73ccf046951","Type":"ContainerStarted","Data":"057768d2f9e4d4b4824e8d96370a7c2f68623a9ccb6cd67fbcaaf7c730a7c81e"} Jul 20 16:15:00.527814 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:00.527646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkt8q" event={"ID":"c4e6ee43-be51-4ce7-9617-25dafef1add4","Type":"ContainerStarted","Data":"962656a0821df261981ec3a46dc7c3ba83f78013156f50417dcc408215dcea6c"} Jul 20 16:15:01.549725 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.548733 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d7857bfc8-5l6mb"] Jul 20 16:15:01.555656 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.555215 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.560703 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.560076 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 16:15:01.560703 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.560307 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 16:15:01.560703 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.560500 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 16:15:01.564331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.563072 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sdtj4\"" Jul 20 16:15:01.564331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.563297 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 16:15:01.564331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.563492 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 16:15:01.564331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.563717 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 16:15:01.564331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.564189 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 16:15:01.571398 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.565446 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d7857bfc8-5l6mb"] Jul 20 16:15:01.733027 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.732772 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-serving-cert\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.733219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.733059 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-service-ca\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.733219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.733098 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnt6h\" (UniqueName: \"kubernetes.io/projected/c4085b90-2f60-4834-90f2-6a0b9bd5d980-kube-api-access-jnt6h\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.733219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.733140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-oauth-config\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.733219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.733182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-config\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.733219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.733212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-oauth-serving-cert\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.834204 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-service-ca\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.834204 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834175 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnt6h\" (UniqueName: \"kubernetes.io/projected/c4085b90-2f60-4834-90f2-6a0b9bd5d980-kube-api-access-jnt6h\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.834405 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-oauth-config\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.834405 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834261 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-config\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.834405 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-oauth-serving-cert\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.834405 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834356 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-serving-cert\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.835034 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.834960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-service-ca\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.835155 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.835125 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-config\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.835227 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.835169 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-oauth-serving-cert\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.838702 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.838675 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-serving-cert\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.842844 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.842821 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnt6h\" (UniqueName: \"kubernetes.io/projected/c4085b90-2f60-4834-90f2-6a0b9bd5d980-kube-api-access-jnt6h\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.843400 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.843376 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-oauth-config\") pod \"console-6d7857bfc8-5l6mb\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:01.874646 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:01.874537 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:02.086998 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:02.086930 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d7857bfc8-5l6mb"] Jul 20 16:15:02.132145 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:02.130712 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4085b90_2f60_4834_90f2_6a0b9bd5d980.slice/crio-c35593a85ea03e273107334395f0826ac2a6fd48b36ec997a04529f90e777a34 WatchSource:0}: Error finding container c35593a85ea03e273107334395f0826ac2a6fd48b36ec997a04529f90e777a34: Status 404 returned error can't find the container with id c35593a85ea03e273107334395f0826ac2a6fd48b36ec997a04529f90e777a34 Jul 20 16:15:02.536523 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:02.536479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkt8q" event={"ID":"c4e6ee43-be51-4ce7-9617-25dafef1add4","Type":"ContainerStarted","Data":"cf570b45b0dbeb8217dc282cf25c23a1a2ce5866d3332248e2f33c83fa88d2dd"} Jul 20 16:15:02.536796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:02.536581 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkt8q" event={"ID":"c4e6ee43-be51-4ce7-9617-25dafef1add4","Type":"ContainerStarted","Data":"f3ec15c29bd649cfd7b7f684bbbae6694f4486adbf6498784142d98c7a0288d4"} Jul 20 16:15:02.536796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:02.536612 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dkt8q" Jul 20 16:15:02.537931 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:02.537897 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7857bfc8-5l6mb" event={"ID":"c4085b90-2f60-4834-90f2-6a0b9bd5d980","Type":"ContainerStarted","Data":"c35593a85ea03e273107334395f0826ac2a6fd48b36ec997a04529f90e777a34"} Jul 20 16:15:02.553599 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:02.553546 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dkt8q" podStartSLOduration=1.187573068 podStartE2EDuration="3.553526876s" totalImagesPullingTime="2.365953808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:02.552225684 +0000 UTC m=+50.757371428" watchObservedRunningTime="2026-07-20 16:15:02.553526876 +0000 UTC m=+50.758672618" Jul 20 16:15:06.550412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:06.550155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ff5cq" event={"ID":"b6fde27d-e433-493b-ab11-b73ccf046951","Type":"ContainerStarted","Data":"3f15547a429b36909d5ce0bf4c83b4a7e44b281e2811cd3d74688724e6988a44"} Jul 20 16:15:06.551687 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:06.551649 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bdxf" event={"ID":"a4851e3b-efcd-4291-8c66-61efff445cc6","Type":"ContainerStarted","Data":"8c4c42159babfda32584720ec66da9a5ce7fd28535ba1f6a096f43af74142f00"} Jul 20 16:15:06.566419 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:06.566377 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ff5cq" podStartSLOduration=1.284387774 podStartE2EDuration="7.566363095s" totalImagesPullingTime="6.281975321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:06.565425209 +0000 UTC m=+54.770570950" watchObservedRunningTime="2026-07-20 16:15:06.566363095 +0000 UTC m=+54.771508835" Jul 20 16:15:07.535134 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.535101 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4"] Jul 20 16:15:07.537352 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.537316 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:07.540049 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.540024 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 20 16:15:07.540191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.540063 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-zp9p8\"" Jul 20 16:15:07.547306 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.547285 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4"] Jul 20 16:15:07.577880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.577847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/69a287c4-7154-4f30-8586-b3516793335a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-6xbs4\" (UID: \"69a287c4-7154-4f30-8586-b3516793335a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:07.678511 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.678470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/69a287c4-7154-4f30-8586-b3516793335a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-6xbs4\" (UID: \"69a287c4-7154-4f30-8586-b3516793335a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:07.682932 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.682882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/69a287c4-7154-4f30-8586-b3516793335a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-6xbs4\" (UID: \"69a287c4-7154-4f30-8586-b3516793335a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:07.848970 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:07.848894 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:10.403434 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:10.403389 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4"] Jul 20 16:15:10.406572 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:10.406548 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a287c4_7154_4f30_8586_b3516793335a.slice/crio-2d30494b0a9350666526cbeadb1c646e9da7f15af8ad0608c00a41f46c29bf05 WatchSource:0}: Error finding container 2d30494b0a9350666526cbeadb1c646e9da7f15af8ad0608c00a41f46c29bf05: Status 404 returned error can't find the container with id 2d30494b0a9350666526cbeadb1c646e9da7f15af8ad0608c00a41f46c29bf05 Jul 20 16:15:10.564361 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:10.564282 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bdxf" event={"ID":"a4851e3b-efcd-4291-8c66-61efff445cc6","Type":"ContainerStarted","Data":"770bce5595a515635265625a86febf4c01ea8d1829088a6cd376ac84ceece1d9"} Jul 20 16:15:10.565386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:10.565360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" event={"ID":"69a287c4-7154-4f30-8586-b3516793335a","Type":"ContainerStarted","Data":"2d30494b0a9350666526cbeadb1c646e9da7f15af8ad0608c00a41f46c29bf05"} Jul 20 16:15:10.566740 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:10.566719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7857bfc8-5l6mb" event={"ID":"c4085b90-2f60-4834-90f2-6a0b9bd5d980","Type":"ContainerStarted","Data":"4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f"} Jul 20 16:15:10.584319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:10.584280 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d7857bfc8-5l6mb" podStartSLOduration=1.4459024839999999 podStartE2EDuration="9.584267333s" totalImagesPullingTime="8.138364849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:10.582976837 +0000 UTC m=+58.788122578" watchObservedRunningTime="2026-07-20 16:15:10.584267333 +0000 UTC m=+58.789413076" Jul 20 16:15:11.129061 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.129029 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d48dcdb85-9j2kh"] Jul 20 16:15:11.130864 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.130849 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.139286 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.139267 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 16:15:11.144760 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.144738 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d48dcdb85-9j2kh"] Jul 20 16:15:11.208177 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208128 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-service-ca\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.208319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-oauth-config\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.208319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208274 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-console-config\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.208319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208299 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-trusted-ca-bundle\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.208454 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-oauth-serving-cert\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.208454 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64hsd\" (UniqueName: \"kubernetes.io/projected/e1e85e25-b114-4867-9360-f7326d96adc2-kube-api-access-64hsd\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.208454 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.208436 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-serving-cert\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.309782 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.309756 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-service-ca\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.309937 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.309824 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-oauth-config\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.309937 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.309846 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-console-config\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.309937 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.309870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-trusted-ca-bundle\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.309937 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.309897 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-oauth-serving-cert\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.311673 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-console-config\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.310052 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64hsd\" (UniqueName: \"kubernetes.io/projected/e1e85e25-b114-4867-9360-f7326d96adc2-kube-api-access-64hsd\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.311814 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-serving-cert\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.312154 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-service-ca\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.312216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-trusted-ca-bundle\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.312689 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-oauth-serving-cert\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.313645 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.313433 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-oauth-config\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.316219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.316199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-serving-cert\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.318886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.318862 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64hsd\" (UniqueName: \"kubernetes.io/projected/e1e85e25-b114-4867-9360-f7326d96adc2-kube-api-access-64hsd\") pod \"console-5d48dcdb85-9j2kh\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.440003 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.439973 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:11.602014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.601987 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d48dcdb85-9j2kh"] Jul 20 16:15:11.607899 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:11.607228 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e85e25_b114_4867_9360_f7326d96adc2.slice/crio-e8f516cf5626ff1a8e9e4d7635b1ed7b56ca111adba683cd63f8b0844edbe0ba WatchSource:0}: Error finding container e8f516cf5626ff1a8e9e4d7635b1ed7b56ca111adba683cd63f8b0844edbe0ba: Status 404 returned error can't find the container with id e8f516cf5626ff1a8e9e4d7635b1ed7b56ca111adba683cd63f8b0844edbe0ba Jul 20 16:15:11.875726 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.875687 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:11.875896 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.875762 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:11.877334 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.877294 2582 patch_prober.go:28] interesting pod/console-6d7857bfc8-5l6mb container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 20 16:15:11.877460 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:11.877354 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6d7857bfc8-5l6mb" podUID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 20 16:15:12.489018 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.488995 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jxd8x" Jul 20 16:15:12.543177 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.543141 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dkt8q" Jul 20 16:15:12.579816 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.579779 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d48dcdb85-9j2kh" event={"ID":"e1e85e25-b114-4867-9360-f7326d96adc2","Type":"ContainerStarted","Data":"ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795"} Jul 20 16:15:12.579981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.579835 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d48dcdb85-9j2kh" event={"ID":"e1e85e25-b114-4867-9360-f7326d96adc2","Type":"ContainerStarted","Data":"e8f516cf5626ff1a8e9e4d7635b1ed7b56ca111adba683cd63f8b0844edbe0ba"} Jul 20 16:15:12.581435 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.581405 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" event={"ID":"69a287c4-7154-4f30-8586-b3516793335a","Type":"ContainerStarted","Data":"742e2a78ef8bc2c26f49049910dc2d530981abf386babdb860983e376e92f7c4"} Jul 20 16:15:12.618244 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.618202 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" podStartSLOduration=3.581957291 podStartE2EDuration="5.618188109s" totalImagesPullingTime="2.036230818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:12.617936318 +0000 UTC m=+60.823082079" watchObservedRunningTime="2026-07-20 16:15:12.618188109 +0000 UTC m=+60.823333847" Jul 20 16:15:12.618370 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:12.618270 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d48dcdb85-9j2kh" podStartSLOduration=1.618265855 podStartE2EDuration="1.618265855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:12.597051141 +0000 UTC m=+60.802196885" watchObservedRunningTime="2026-07-20 16:15:12.618265855 +0000 UTC m=+60.823411596" Jul 20 16:15:13.584833 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:13.584801 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:13.590915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:13.590889 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-6xbs4" Jul 20 16:15:14.591258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:14.591174 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bdxf" event={"ID":"a4851e3b-efcd-4291-8c66-61efff445cc6","Type":"ContainerStarted","Data":"a90fbe46733a12285ba90996452b11293326cffcb299d99e44c2984e7f481c66"} Jul 20 16:15:14.612091 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:14.612047 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4bdxf" podStartSLOduration=1.2639593310000001 podStartE2EDuration="15.61203267s" totalImagesPullingTime="14.348073339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:14.610646291 +0000 UTC m=+62.815792058" watchObservedRunningTime="2026-07-20 16:15:14.61203267 +0000 UTC m=+62.817178389" Jul 20 16:15:17.967589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:17.967547 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:15:17.970146 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:17.970128 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 16:15:17.980875 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:17.980855 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 16:15:17.991215 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:17.991192 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5xnz\" (UniqueName: \"kubernetes.io/projected/2b52bcb1-7c46-4c77-a949-d5d261f243ad-kube-api-access-g5xnz\") pod \"network-check-target-xswbf\" (UID: \"2b52bcb1-7c46-4c77-a949-d5d261f243ad\") " pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:15:18.068143 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.068094 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:15:18.068345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.068163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:15:18.070768 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.070747 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 16:15:18.070888 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.070747 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 16:15:18.080987 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.080965 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4-original-pull-secret\") pod \"global-pull-secret-syncer-szff5\" (UID: \"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4\") " pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:15:18.081092 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.080967 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f70698c0-e1fe-42e1-a111-cacbd6efcddd-metrics-certs\") pod \"network-metrics-daemon-6vk8k\" (UID: \"f70698c0-e1fe-42e1-a111-cacbd6efcddd\") " pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:15:18.230804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.230726 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szff5" Jul 20 16:15:18.238763 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.238736 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fbfhg\"" Jul 20 16:15:18.242863 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.242844 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sngxb\"" Jul 20 16:15:18.247220 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.247199 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6vk8k" Jul 20 16:15:18.251632 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.251526 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:15:18.369833 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.369747 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-szff5"] Jul 20 16:15:18.372904 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:18.372868 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dade5e1_3c11_4fd3_8c58_3dad3bf4f1a4.slice/crio-95806028c6b74e13ca9d315cbff12b125775473142d0ec2d824db471459290e8 WatchSource:0}: Error finding container 95806028c6b74e13ca9d315cbff12b125775473142d0ec2d824db471459290e8: Status 404 returned error can't find the container with id 95806028c6b74e13ca9d315cbff12b125775473142d0ec2d824db471459290e8 Jul 20 16:15:18.386595 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.386569 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6vk8k"] Jul 20 16:15:18.389724 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:18.389612 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70698c0_e1fe_42e1_a111_cacbd6efcddd.slice/crio-63257cf2a8a98d0d7dd2444a5130193e49e63beaccd3182a88851b34d946b0cf WatchSource:0}: Error finding container 63257cf2a8a98d0d7dd2444a5130193e49e63beaccd3182a88851b34d946b0cf: Status 404 returned error can't find the container with id 63257cf2a8a98d0d7dd2444a5130193e49e63beaccd3182a88851b34d946b0cf Jul 20 16:15:18.406571 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.406553 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xswbf"] Jul 20 16:15:18.408787 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:18.408760 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b52bcb1_7c46_4c77_a949_d5d261f243ad.slice/crio-aea1f2756eb682c72e66f6babe133fd1a6f33a556f539d0a4043e80538bf2108 WatchSource:0}: Error finding container aea1f2756eb682c72e66f6babe133fd1a6f33a556f539d0a4043e80538bf2108: Status 404 returned error can't find the container with id aea1f2756eb682c72e66f6babe133fd1a6f33a556f539d0a4043e80538bf2108 Jul 20 16:15:18.606657 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.606568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xswbf" event={"ID":"2b52bcb1-7c46-4c77-a949-d5d261f243ad","Type":"ContainerStarted","Data":"aea1f2756eb682c72e66f6babe133fd1a6f33a556f539d0a4043e80538bf2108"} Jul 20 16:15:18.607661 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.607631 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6vk8k" event={"ID":"f70698c0-e1fe-42e1-a111-cacbd6efcddd","Type":"ContainerStarted","Data":"63257cf2a8a98d0d7dd2444a5130193e49e63beaccd3182a88851b34d946b0cf"} Jul 20 16:15:18.608581 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:18.608560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-szff5" event={"ID":"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4","Type":"ContainerStarted","Data":"95806028c6b74e13ca9d315cbff12b125775473142d0ec2d824db471459290e8"} Jul 20 16:15:20.620176 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:20.619978 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6vk8k" event={"ID":"f70698c0-e1fe-42e1-a111-cacbd6efcddd","Type":"ContainerStarted","Data":"409f1a6ffb21d327d759586df610e94c6d220b206bfad4b1709c3e3aee27f528"} Jul 20 16:15:20.620176 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:20.620054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6vk8k" event={"ID":"f70698c0-e1fe-42e1-a111-cacbd6efcddd","Type":"ContainerStarted","Data":"c91818b793b0b0af3df9973b68b189e216ae76cbe176340c842e921edefcdb07"} Jul 20 16:15:20.636293 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:20.636236 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6vk8k" podStartSLOduration=66.69170504 podStartE2EDuration="1m8.636221022s" totalImagesPullingTime="1.944515982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:20.63519383 +0000 UTC m=+68.840339573" watchObservedRunningTime="2026-07-20 16:15:20.636221022 +0000 UTC m=+68.841366764" Jul 20 16:15:21.440590 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:21.440554 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:21.440765 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:21.440636 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:21.446195 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:21.446172 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:21.628988 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:21.628959 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:15:21.671025 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:21.670642 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d7857bfc8-5l6mb"] Jul 20 16:15:22.917080 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.917046 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h"] Jul 20 16:15:22.919358 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.919340 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:22.922161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.922122 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 16:15:22.922275 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.922161 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 16:15:22.922275 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.922266 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 16:15:22.923274 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.923251 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 16:15:22.923274 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.923265 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8ll4x\"" Jul 20 16:15:22.923401 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.923318 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 16:15:22.935133 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.935111 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h"] Jul 20 16:15:22.958354 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.958323 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g22m4"] Jul 20 16:15:22.960791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.960767 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:22.963365 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.963331 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 16:15:22.963365 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.963360 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 16:15:22.963558 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.963364 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t4pfw\"" Jul 20 16:15:22.963558 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:22.963346 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 16:15:23.005675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqc72\" (UniqueName: \"kubernetes.io/projected/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-kube-api-access-sqc72\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.005924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-wtmp\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.005924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005792 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.005924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.005924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-accelerators-collector-config\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.005924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005861 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e661263-e149-45a6-b34a-fa68e4bd7282-metrics-client-ca\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.005924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005915 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.006183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-sys\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.006183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.005958 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-textfile\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.006183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.006038 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-root\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.006183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.006072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-tls\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.006183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.006097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.006183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.006118 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkj68\" (UniqueName: \"kubernetes.io/projected/1e661263-e149-45a6-b34a-fa68e4bd7282-kube-api-access-jkj68\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107129 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e661263-e149-45a6-b34a-fa68e4bd7282-metrics-client-ca\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107139 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.107281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-sys\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107224 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-sys\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-textfile\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107410 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-root\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107410 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-tls\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107401 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-root\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107544 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:15:23.107484 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 16:15:23.107593 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:15:23.107545 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-tls podName:1e661263-e149-45a6-b34a-fa68e4bd7282 nodeName:}" failed. No retries permitted until 2026-07-20 16:15:23.607526288 +0000 UTC m=+71.812672009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-tls") pod "node-exporter-g22m4" (UID: "1e661263-e149-45a6-b34a-fa68e4bd7282") : secret "node-exporter-tls" not found Jul 20 16:15:23.107593 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkj68\" (UniqueName: \"kubernetes.io/projected/1e661263-e149-45a6-b34a-fa68e4bd7282-kube-api-access-jkj68\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107730 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107574 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-textfile\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107730 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqc72\" (UniqueName: \"kubernetes.io/projected/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-kube-api-access-sqc72\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.107730 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-wtmp\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e661263-e149-45a6-b34a-fa68e4bd7282-metrics-client-ca\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.107880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107761 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.107880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.107880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-accelerators-collector-config\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.108073 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:15:23.107887 2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 20 16:15:23.108073 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.107907 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-wtmp\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.108073 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:15:23.107950 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-tls podName:1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c nodeName:}" failed. No retries permitted until 2026-07-20 16:15:23.607931657 +0000 UTC m=+71.813077377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-2rz9h" (UID: "1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c") : secret "openshift-state-metrics-tls" not found Jul 20 16:15:23.108357 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.108337 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-accelerators-collector-config\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.108496 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.108476 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.111677 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.109843 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.111677 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.110211 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.118901 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.118876 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkj68\" (UniqueName: \"kubernetes.io/projected/1e661263-e149-45a6-b34a-fa68e4bd7282-kube-api-access-jkj68\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.119188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.119166 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqc72\" (UniqueName: \"kubernetes.io/projected/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-kube-api-access-sqc72\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.612209 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.612125 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.612209 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.612176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-tls\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.614372 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.614350 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e661263-e149-45a6-b34a-fa68e4bd7282-node-exporter-tls\") pod \"node-exporter-g22m4\" (UID: \"1e661263-e149-45a6-b34a-fa68e4bd7282\") " pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.614492 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.614474 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-2rz9h\" (UID: \"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.632323 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.632298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xswbf" event={"ID":"2b52bcb1-7c46-4c77-a949-d5d261f243ad","Type":"ContainerStarted","Data":"0264c2093d4062878799fa34d66b9f46cb9274205f23bc42b1780b44059b0aa8"} Jul 20 16:15:23.632423 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.632408 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:15:23.633788 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.633767 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-szff5" event={"ID":"6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4","Type":"ContainerStarted","Data":"778589efde768fcdb9851f15cdaaec250bf709f1fe5afc964adb3e0b658e4bf3"} Jul 20 16:15:23.647872 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.647833 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xswbf" podStartSLOduration=66.930453871 podStartE2EDuration="1m11.647821984s" totalImagesPullingTime="4.717368113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:23.646807812 +0000 UTC m=+71.851953554" watchObservedRunningTime="2026-07-20 16:15:23.647821984 +0000 UTC m=+71.852967726" Jul 20 16:15:23.660704 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.660659 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-szff5" podStartSLOduration=66.905751452 podStartE2EDuration="1m11.660644618s" totalImagesPullingTime="4.754893166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:23.660019907 +0000 UTC m=+71.865165649" watchObservedRunningTime="2026-07-20 16:15:23.660644618 +0000 UTC m=+71.865790357" Jul 20 16:15:23.829293 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.829269 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" Jul 20 16:15:23.870909 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.870833 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g22m4" Jul 20 16:15:23.889000 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:23.888958 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e661263_e149_45a6_b34a_fa68e4bd7282.slice/crio-bbe736163a5f9157a7adfb9bc9483240a11e5e0144bb58aa03aa890e9865c847 WatchSource:0}: Error finding container bbe736163a5f9157a7adfb9bc9483240a11e5e0144bb58aa03aa890e9865c847: Status 404 returned error can't find the container with id bbe736163a5f9157a7adfb9bc9483240a11e5e0144bb58aa03aa890e9865c847 Jul 20 16:15:23.951555 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:23.951526 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h"] Jul 20 16:15:23.954191 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:23.954162 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccfa788_fb2e_4d7a_9fbb_66c1c51c8d3c.slice/crio-9108d913b45a4f871c78f7fc1200663118f85aa5321bd070fb1da899491a8797 WatchSource:0}: Error finding container 9108d913b45a4f871c78f7fc1200663118f85aa5321bd070fb1da899491a8797: Status 404 returned error can't find the container with id 9108d913b45a4f871c78f7fc1200663118f85aa5321bd070fb1da899491a8797 Jul 20 16:15:24.005108 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.005091 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:15:24.008873 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.008858 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.011188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011167 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 16:15:24.011188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011174 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 16:15:24.011567 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011415 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 16:15:24.011567 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011455 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wh9xm\"" Jul 20 16:15:24.011567 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011508 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 16:15:24.011567 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011558 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 16:15:24.011863 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011669 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 16:15:24.011863 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011802 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 16:15:24.011863 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.011836 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 16:15:24.017943 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.017916 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 16:15:24.025313 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.025286 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:15:24.114802 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114774 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-web-config\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114810 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-config-volume\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114867 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdlsg\" (UniqueName: \"kubernetes.io/projected/1033d9d8-d648-4d33-ad6b-4f8b5f086566-kube-api-access-kdlsg\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.114934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1033d9d8-d648-4d33-ad6b-4f8b5f086566-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.115190 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.114968 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.115190 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.115001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1033d9d8-d648-4d33-ad6b-4f8b5f086566-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.115190 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.115022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1033d9d8-d648-4d33-ad6b-4f8b5f086566-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.115190 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.115044 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1033d9d8-d648-4d33-ad6b-4f8b5f086566-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.115190 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.115064 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1033d9d8-d648-4d33-ad6b-4f8b5f086566-config-out\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216336 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216486 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-config-volume\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216486 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216376 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216594 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdlsg\" (UniqueName: \"kubernetes.io/projected/1033d9d8-d648-4d33-ad6b-4f8b5f086566-kube-api-access-kdlsg\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216594 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216594 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216769 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216597 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1033d9d8-d648-4d33-ad6b-4f8b5f086566-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216769 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216769 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216683 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1033d9d8-d648-4d33-ad6b-4f8b5f086566-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216769 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1033d9d8-d648-4d33-ad6b-4f8b5f086566-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.216769 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216753 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1033d9d8-d648-4d33-ad6b-4f8b5f086566-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.217022 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216780 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1033d9d8-d648-4d33-ad6b-4f8b5f086566-config-out\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.217022 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.216823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-web-config\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.217155 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.217132 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1033d9d8-d648-4d33-ad6b-4f8b5f086566-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.218016 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.217986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1033d9d8-d648-4d33-ad6b-4f8b5f086566-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.218331 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.218297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1033d9d8-d648-4d33-ad6b-4f8b5f086566-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.219381 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.219361 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.219530 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.219510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.219825 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.219802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-web-config\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.219977 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.219808 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-config-volume\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.220184 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.220160 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.220263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.220186 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.220472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.220455 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1033d9d8-d648-4d33-ad6b-4f8b5f086566-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.221249 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.221231 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1033d9d8-d648-4d33-ad6b-4f8b5f086566-config-out\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.221325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.221235 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1033d9d8-d648-4d33-ad6b-4f8b5f086566-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.223213 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.223196 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdlsg\" (UniqueName: \"kubernetes.io/projected/1033d9d8-d648-4d33-ad6b-4f8b5f086566-kube-api-access-kdlsg\") pod \"alertmanager-main-0\" (UID: \"1033d9d8-d648-4d33-ad6b-4f8b5f086566\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.321630 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.321588 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:15:24.450186 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.450155 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:15:24.453158 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:24.453134 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1033d9d8_d648_4d33_ad6b_4f8b5f086566.slice/crio-e6824c483cee94ab205d83c605d13ce018f336340a7e73c5a37d93768292bb11 WatchSource:0}: Error finding container e6824c483cee94ab205d83c605d13ce018f336340a7e73c5a37d93768292bb11: Status 404 returned error can't find the container with id e6824c483cee94ab205d83c605d13ce018f336340a7e73c5a37d93768292bb11 Jul 20 16:15:24.639052 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.638983 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"e6824c483cee94ab205d83c605d13ce018f336340a7e73c5a37d93768292bb11"} Jul 20 16:15:24.640138 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.640115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g22m4" event={"ID":"1e661263-e149-45a6-b34a-fa68e4bd7282","Type":"ContainerStarted","Data":"bbe736163a5f9157a7adfb9bc9483240a11e5e0144bb58aa03aa890e9865c847"} Jul 20 16:15:24.641879 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.641858 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" event={"ID":"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c","Type":"ContainerStarted","Data":"240e106b8c331653ec8908535d7987a0b2e6835f29d9474964e99e2ff03af013"} Jul 20 16:15:24.641935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.641904 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" event={"ID":"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c","Type":"ContainerStarted","Data":"bf79c7b6021202e5fad4218b4d80b7f3df590815acee4bda258420d8b01b6f37"} Jul 20 16:15:24.641935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:24.641922 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" event={"ID":"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c","Type":"ContainerStarted","Data":"9108d913b45a4f871c78f7fc1200663118f85aa5321bd070fb1da899491a8797"} Jul 20 16:15:25.001113 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.001078 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5cd8845757-9lkwc"] Jul 20 16:15:25.004556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.004540 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.007271 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.007248 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 20 16:15:25.007697 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.007681 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-m6qkf\"" Jul 20 16:15:25.008084 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.008064 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-97asdfhd487bk\"" Jul 20 16:15:25.008165 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.008116 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 20 16:15:25.008340 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.008322 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 20 16:15:25.008451 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.008396 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 20 16:15:25.008553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.008538 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 20 16:15:25.019804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.019709 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cd8845757-9lkwc"] Jul 20 16:15:25.125957 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.125907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126126 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.125959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-metrics-client-ca\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126126 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.125997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126126 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.126027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126126 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.126055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126338 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.126126 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-tls\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126338 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.126182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5cs2\" (UniqueName: \"kubernetes.io/projected/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-kube-api-access-n5cs2\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.126338 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.126233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-grpc-tls\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227090 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227268 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227099 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227268 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227268 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-tls\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227268 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cs2\" (UniqueName: \"kubernetes.io/projected/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-kube-api-access-n5cs2\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227268 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-grpc-tls\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.227520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.227329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-metrics-client-ca\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.228142 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.228094 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-metrics-client-ca\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.230633 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.230592 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.230805 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.230781 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.231092 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.231070 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.231186 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.231106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-tls\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.231328 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.231307 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-grpc-tls\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.231587 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.231563 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.235597 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.235574 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cs2\" (UniqueName: \"kubernetes.io/projected/810cf5e9-67a5-46b8-9158-c4eb638b3ef0-kube-api-access-n5cs2\") pod \"thanos-querier-5cd8845757-9lkwc\" (UID: \"810cf5e9-67a5-46b8-9158-c4eb638b3ef0\") " pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.314960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.314929 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:25.481165 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.481059 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cd8845757-9lkwc"] Jul 20 16:15:25.583005 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:25.582929 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod810cf5e9_67a5_46b8_9158_c4eb638b3ef0.slice/crio-8d15f32f5bf1a9122afb641a76a5f48fd66fe7110fa01c91d49f8f5e20e90c57 WatchSource:0}: Error finding container 8d15f32f5bf1a9122afb641a76a5f48fd66fe7110fa01c91d49f8f5e20e90c57: Status 404 returned error can't find the container with id 8d15f32f5bf1a9122afb641a76a5f48fd66fe7110fa01c91d49f8f5e20e90c57 Jul 20 16:15:25.645677 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.645650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"8d15f32f5bf1a9122afb641a76a5f48fd66fe7110fa01c91d49f8f5e20e90c57"} Jul 20 16:15:25.647167 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.647144 2582 generic.go:356] "Generic (PLEG): container finished" podID="1e661263-e149-45a6-b34a-fa68e4bd7282" containerID="84a27d9bf13a466e7992cb5f83af5a7c2f3e18d312bc75bc7bfac72b7d8f8e58" exitCode=0 Jul 20 16:15:25.647250 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:25.647186 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g22m4" event={"ID":"1e661263-e149-45a6-b34a-fa68e4bd7282","Type":"ContainerDied","Data":"84a27d9bf13a466e7992cb5f83af5a7c2f3e18d312bc75bc7bfac72b7d8f8e58"} Jul 20 16:15:26.655125 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:26.655087 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g22m4" event={"ID":"1e661263-e149-45a6-b34a-fa68e4bd7282","Type":"ContainerStarted","Data":"58878f57b0616883c74bf8bc9506b0a4890ccea9d704163e25e159f4fc7ddbee"} Jul 20 16:15:26.655578 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:26.655149 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g22m4" event={"ID":"1e661263-e149-45a6-b34a-fa68e4bd7282","Type":"ContainerStarted","Data":"743d29055bd18412dd015572d0bed4c89156b65068f1c1db50eb34ec5845b9e3"} Jul 20 16:15:26.657764 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:26.657738 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" event={"ID":"1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c","Type":"ContainerStarted","Data":"30c4bd9232706d39af38382f941ce8357a1360efdadbab456254b520cff5d1a1"} Jul 20 16:15:26.675787 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:26.675742 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g22m4" podStartSLOduration=3.212858358 podStartE2EDuration="4.67572899s" totalImagesPullingTime="1.36477635s" totalInitContainerRuntime="98.094282ms" isStatefulPod=true podCreationTimestamp="2026-07-20 16:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:26.673782796 +0000 UTC m=+74.878928541" watchObservedRunningTime="2026-07-20 16:15:26.67572899 +0000 UTC m=+74.880874732" Jul 20 16:15:26.690673 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:26.690600 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-2rz9h" podStartSLOduration=3.121662876 podStartE2EDuration="4.690581479s" totalImagesPullingTime="1.568918603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:26.689724293 +0000 UTC m=+74.894870033" watchObservedRunningTime="2026-07-20 16:15:26.690581479 +0000 UTC m=+74.895727352" Jul 20 16:15:27.664395 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:27.664353 2582 generic.go:356] "Generic (PLEG): container finished" podID="1033d9d8-d648-4d33-ad6b-4f8b5f086566" containerID="39a5debc3f74ee030f6b04d963c5b8a00937a17f7ef2a183d14772a8c6fbd66f" exitCode=0 Jul 20 16:15:27.664847 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:27.664424 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerDied","Data":"39a5debc3f74ee030f6b04d963c5b8a00937a17f7ef2a183d14772a8c6fbd66f"} Jul 20 16:15:28.669948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:28.669913 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"caeaf9c7d4db62f657235e6e5ce8cab8295256c07563fbf6f575021a91d9b0fb"} Jul 20 16:15:28.670308 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:28.669985 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"7d2c8e9b3289cda3d833d1f75e5eb40c3b0294b7aab89d77ca524d94c8efdd0e"} Jul 20 16:15:28.670308 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:28.670010 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"e0437de6af6bcf818f151f2a9f93d8c4d6d0acc62b4527fc712ef78d2dc53f5d"} Jul 20 16:15:29.109520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.109486 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:15:29.113829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.113805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.116540 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.116509 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d4fvk\"" Jul 20 16:15:29.116721 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.116582 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 20 16:15:29.116721 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.116517 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 20 16:15:29.117011 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.116989 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 20 16:15:29.117132 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.117077 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 20 16:15:29.117567 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.117547 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 20 16:15:29.117700 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.117651 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cl16a1ig68eur\"" Jul 20 16:15:29.117700 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.117687 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 16:15:29.117885 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.117838 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 20 16:15:29.118243 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.118222 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 20 16:15:29.118317 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.118293 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 20 16:15:29.119260 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.119232 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 20 16:15:29.121038 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.120526 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 20 16:15:29.121038 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.120865 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 20 16:15:29.121191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.121149 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 20 16:15:29.128370 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.127864 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 20 16:15:29.129665 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.129538 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:15:29.133228 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.133208 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 20 16:15:29.160008 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.159929 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160008 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.159970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160008 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.159997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5kh6\" (UniqueName: \"kubernetes.io/projected/67e74dca-e0cf-4fb0-96bc-460464a09573-kube-api-access-w5kh6\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-web-config\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160239 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67e74dca-e0cf-4fb0-96bc-460464a09573-config-out\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160310 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67e74dca-e0cf-4fb0-96bc-460464a09573-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160426 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160472 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160468 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160710 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160505 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160710 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160537 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-config\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160710 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160560 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160710 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.160710 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.160659 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261334 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5kh6\" (UniqueName: \"kubernetes.io/projected/67e74dca-e0cf-4fb0-96bc-460464a09573-kube-api-access-w5kh6\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261461 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261891 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261891 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261800 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-web-config\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261891 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.261891 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.262077 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.261902 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67e74dca-e0cf-4fb0-96bc-460464a09573-config-out\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.262224 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.262203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.262333 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.262310 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.262916 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.262746 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.262916 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.262757 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67e74dca-e0cf-4fb0-96bc-460464a09573-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.262916 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.262812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.263122 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263014 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.263122 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263084 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.264261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.264261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.264261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263381 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.264261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263415 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-config\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.264261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.263441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.264261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.264156 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.265120 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.264753 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.265859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.265603 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.265859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.265818 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67e74dca-e0cf-4fb0-96bc-460464a09573-config-out\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.266915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.266034 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67e74dca-e0cf-4fb0-96bc-460464a09573-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.266915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.266446 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.266915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.266817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.266915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.266870 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.267721 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.267467 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-web-config\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.267818 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.267759 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.268817 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.268791 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.268909 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.268847 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67e74dca-e0cf-4fb0-96bc-460464a09573-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.269387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.269348 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5kh6\" (UniqueName: \"kubernetes.io/projected/67e74dca-e0cf-4fb0-96bc-460464a09573-kube-api-access-w5kh6\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.270222 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.270197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.270315 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.270263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/67e74dca-e0cf-4fb0-96bc-460464a09573-config\") pod \"prometheus-k8s-0\" (UID: \"67e74dca-e0cf-4fb0-96bc-460464a09573\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.351991 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.351959 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fb97fb888-ltwkd"] Jul 20 16:15:29.356544 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.356521 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.364081 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.363997 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb97fb888-ltwkd"] Jul 20 16:15:29.431225 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.431194 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:29.465116 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465089 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-serving-cert\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.465280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-service-ca\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.465280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465156 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-console-config\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.465280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-oauth-config\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.465424 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465288 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-trusted-ca-bundle\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.465424 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465304 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22gsb\" (UniqueName: \"kubernetes.io/projected/80507aae-8478-4d34-8daf-6feed87c6d63-kube-api-access-22gsb\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.465424 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.465377 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-oauth-serving-cert\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.565997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-service-ca\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-console-config\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566105 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-oauth-config\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-trusted-ca-bundle\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566166 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22gsb\" (UniqueName: \"kubernetes.io/projected/80507aae-8478-4d34-8daf-6feed87c6d63-kube-api-access-22gsb\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-oauth-serving-cert\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566245 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-serving-cert\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.567031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.566900 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-service-ca\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.567943 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.567724 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-console-config\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.567943 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.567907 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-oauth-serving-cert\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.568802 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.568753 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-trusted-ca-bundle\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.569237 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.569213 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:15:29.570860 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.570829 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-serving-cert\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.571047 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:29.571026 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e74dca_e0cf_4fb0_96bc_460464a09573.slice/crio-1dad9c795c6ecb8d4c0ff52a8360239872b2f682de5e0fa83ec809b6e953abfd WatchSource:0}: Error finding container 1dad9c795c6ecb8d4c0ff52a8360239872b2f682de5e0fa83ec809b6e953abfd: Status 404 returned error can't find the container with id 1dad9c795c6ecb8d4c0ff52a8360239872b2f682de5e0fa83ec809b6e953abfd Jul 20 16:15:29.571326 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.571302 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-oauth-config\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.575554 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.575533 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22gsb\" (UniqueName: \"kubernetes.io/projected/80507aae-8478-4d34-8daf-6feed87c6d63-kube-api-access-22gsb\") pod \"console-fb97fb888-ltwkd\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.668817 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.668793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:29.679325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.679145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"cec0b11f70c2fd056007d7fc9e39cef9953c382a3e47fc5b1f47ecea05836b80"} Jul 20 16:15:29.679325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.679228 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"a42b310455adb65fcf28bf043a4fd7adcdd90dad9848bd9cd7f0107f71bb3ee3"} Jul 20 16:15:29.679325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.679276 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" event={"ID":"810cf5e9-67a5-46b8-9158-c4eb638b3ef0","Type":"ContainerStarted","Data":"ee879c765e4abd540a5ec40cd7de1d760f78b7b00698384e419b5ce3d9428f2d"} Jul 20 16:15:29.679808 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.679705 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:29.682042 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.681717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"30c65224b1353088ad420af8b09285682fb4a5e40aabb85715109ea6a338cc41"} Jul 20 16:15:29.682042 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.681790 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"1dad9c795c6ecb8d4c0ff52a8360239872b2f682de5e0fa83ec809b6e953abfd"} Jul 20 16:15:29.701611 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.701013 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" podStartSLOduration=2.001741089 podStartE2EDuration="5.700992386s" totalImagesPullingTime="3.699251297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:29.700160905 +0000 UTC m=+77.905306648" watchObservedRunningTime="2026-07-20 16:15:29.700992386 +0000 UTC m=+77.906138133" Jul 20 16:15:29.816084 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:29.816051 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb97fb888-ltwkd"] Jul 20 16:15:29.821559 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:15:29.821528 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80507aae_8478_4d34_8daf_6feed87c6d63.slice/crio-cbef3fbf36144e7f9630501c220c6df892380186b0b6a7991d08ef730ac8d6af WatchSource:0}: Error finding container cbef3fbf36144e7f9630501c220c6df892380186b0b6a7991d08ef730ac8d6af: Status 404 returned error can't find the container with id cbef3fbf36144e7f9630501c220c6df892380186b0b6a7991d08ef730ac8d6af Jul 20 16:15:30.689435 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.689397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"31773c163366ee96bc6ac9d6384402fc58cbfd72d1f18c12e9f9b7c4faa5cb7c"} Jul 20 16:15:30.689920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.689474 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"33e272d1e54e8b5984ed08f09bff062b9965bc949f2adfea1b7a61089db2e1a0"} Jul 20 16:15:30.689920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.689518 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"ec96af0f13552cb7e4e6732461c8e5dbfdfaf4edf6e65bae4ccc11350aa09399"} Jul 20 16:15:30.689920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.689565 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"791efdbb75f64bd6a8592545c9c7d982096c71960ba91026152cfcc05b57f5cf"} Jul 20 16:15:30.689920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.689604 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"8bbae0e00b9fc66172a1a5622cf4a856aa752d5f954f186fcaa0ecaf7a3acacd"} Jul 20 16:15:30.689920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.689665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1033d9d8-d648-4d33-ad6b-4f8b5f086566","Type":"ContainerStarted","Data":"99969107d65d0fed230ebd2e0853954dd590e3db7f0138e4725b5b2f95acfc59"} Jul 20 16:15:30.690971 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.690947 2582 generic.go:356] "Generic (PLEG): container finished" podID="67e74dca-e0cf-4fb0-96bc-460464a09573" containerID="30c65224b1353088ad420af8b09285682fb4a5e40aabb85715109ea6a338cc41" exitCode=0 Jul 20 16:15:30.691061 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.690994 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerDied","Data":"30c65224b1353088ad420af8b09285682fb4a5e40aabb85715109ea6a338cc41"} Jul 20 16:15:30.692601 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.692535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb97fb888-ltwkd" event={"ID":"80507aae-8478-4d34-8daf-6feed87c6d63","Type":"ContainerStarted","Data":"c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec"} Jul 20 16:15:30.692601 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.692567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb97fb888-ltwkd" event={"ID":"80507aae-8478-4d34-8daf-6feed87c6d63","Type":"ContainerStarted","Data":"cbef3fbf36144e7f9630501c220c6df892380186b0b6a7991d08ef730ac8d6af"} Jul 20 16:15:30.714994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.714950 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.848666122 podStartE2EDuration="7.714937253s" totalImagesPullingTime="4.848249334s" totalInitContainerRuntime="18.021797ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:30.713052272 +0000 UTC m=+78.918198015" watchObservedRunningTime="2026-07-20 16:15:30.714937253 +0000 UTC m=+78.920082994" Jul 20 16:15:30.729760 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:30.729712 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fb97fb888-ltwkd" podStartSLOduration=1.7296931880000002 podStartE2EDuration="1.729693188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:30.729390111 +0000 UTC m=+78.934535855" watchObservedRunningTime="2026-07-20 16:15:30.729693188 +0000 UTC m=+78.934838931" Jul 20 16:15:34.718328 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.718293 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"059fe261c25684d7eef28246d823bac5ec2f557bba7a10c2bb3e94f5c41c06ae"} Jul 20 16:15:34.718762 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.718356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"3d6854a1d7e3f12c5ed68e1ac634a9093feecf5c20fbcc7b944ac324298b8316"} Jul 20 16:15:34.718762 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.718391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"5429ddaccc09e9325e851e312ffec96e2303532663b6cfe3560eddda3a6806d9"} Jul 20 16:15:34.718762 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.718417 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"9453aad2150eddc04d9a3d293d447b352187dd70761bc89bfa643d24b8cd0115"} Jul 20 16:15:34.718762 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.718440 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"08f664e0a833510e4e1967ea1780ae27dc653244667a1582e7602c335ed9bdff"} Jul 20 16:15:34.718762 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.718475 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e74dca-e0cf-4fb0-96bc-460464a09573","Type":"ContainerStarted","Data":"a429bae0e43eb95ed4077712ca0e799ca35d21863bb119fc6a96302f5f38e243"} Jul 20 16:15:34.748587 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:34.748541 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.5109385140000002 podStartE2EDuration="5.748525014s" totalImagesPullingTime="3.216015357s" totalInitContainerRuntime="21.571143ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:15:34.746040144 +0000 UTC m=+82.951185910" watchObservedRunningTime="2026-07-20 16:15:34.748525014 +0000 UTC m=+82.953670769" Jul 20 16:15:35.700715 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:35.700685 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5cd8845757-9lkwc" Jul 20 16:15:39.431888 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:39.431856 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:15:39.670031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:39.670000 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:39.670031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:39.670047 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:39.674814 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:39.674791 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:39.741925 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:39.741838 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:15:39.785169 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:39.785140 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d48dcdb85-9j2kh"] Jul 20 16:15:46.696575 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:46.696528 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d7857bfc8-5l6mb" podUID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" containerName="console" containerID="cri-o://4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f" gracePeriod=15 Jul 20 16:15:46.931760 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:46.931735 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:47.020492 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020416 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-config\") pod \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " Jul 20 16:15:47.020492 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020481 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-serving-cert\") pod \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " Jul 20 16:15:47.020722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020503 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-oauth-config\") pod \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " Jul 20 16:15:47.020722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020549 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnt6h\" (UniqueName: \"kubernetes.io/projected/c4085b90-2f60-4834-90f2-6a0b9bd5d980-kube-api-access-jnt6h\") pod \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " Jul 20 16:15:47.020722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020573 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-oauth-serving-cert\") pod \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " Jul 20 16:15:47.020722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020613 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-service-ca\") pod \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\" (UID: \"c4085b90-2f60-4834-90f2-6a0b9bd5d980\") " Jul 20 16:15:47.020958 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.020932 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-config" (OuterVolumeSpecName: "console-config") pod "c4085b90-2f60-4834-90f2-6a0b9bd5d980" (UID: "c4085b90-2f60-4834-90f2-6a0b9bd5d980"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:15:47.021071 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.021042 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4085b90-2f60-4834-90f2-6a0b9bd5d980" (UID: "c4085b90-2f60-4834-90f2-6a0b9bd5d980"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:15:47.021188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.021074 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4085b90-2f60-4834-90f2-6a0b9bd5d980" (UID: "c4085b90-2f60-4834-90f2-6a0b9bd5d980"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:15:47.022873 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.022848 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4085b90-2f60-4834-90f2-6a0b9bd5d980-kube-api-access-jnt6h" (OuterVolumeSpecName: "kube-api-access-jnt6h") pod "c4085b90-2f60-4834-90f2-6a0b9bd5d980" (UID: "c4085b90-2f60-4834-90f2-6a0b9bd5d980"). InnerVolumeSpecName "kube-api-access-jnt6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:15:47.022975 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.022853 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4085b90-2f60-4834-90f2-6a0b9bd5d980" (UID: "c4085b90-2f60-4834-90f2-6a0b9bd5d980"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:15:47.022975 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.022917 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4085b90-2f60-4834-90f2-6a0b9bd5d980" (UID: "c4085b90-2f60-4834-90f2-6a0b9bd5d980"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:15:47.121729 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.121695 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:15:47.121729 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.121722 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-oauth-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:15:47.121729 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.121731 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnt6h\" (UniqueName: \"kubernetes.io/projected/c4085b90-2f60-4834-90f2-6a0b9bd5d980-kube-api-access-jnt6h\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:15:47.121945 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.121740 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-oauth-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:15:47.121945 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.121749 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-service-ca\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:15:47.121945 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.121758 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4085b90-2f60-4834-90f2-6a0b9bd5d980-console-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:15:47.771111 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.771076 2582 generic.go:356] "Generic (PLEG): container finished" podID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" containerID="4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f" exitCode=0 Jul 20 16:15:47.771561 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.771128 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7857bfc8-5l6mb" Jul 20 16:15:47.771561 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.771158 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7857bfc8-5l6mb" event={"ID":"c4085b90-2f60-4834-90f2-6a0b9bd5d980","Type":"ContainerDied","Data":"4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f"} Jul 20 16:15:47.771561 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.771214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7857bfc8-5l6mb" event={"ID":"c4085b90-2f60-4834-90f2-6a0b9bd5d980","Type":"ContainerDied","Data":"c35593a85ea03e273107334395f0826ac2a6fd48b36ec997a04529f90e777a34"} Jul 20 16:15:47.771561 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.771241 2582 scope.go:117] "RemoveContainer" containerID="4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f" Jul 20 16:15:47.781571 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.781554 2582 scope.go:117] "RemoveContainer" containerID="4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f" Jul 20 16:15:47.781879 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:15:47.781852 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f\": container with ID starting with 4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f not found: ID does not exist" containerID="4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f" Jul 20 16:15:47.781939 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.781883 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f"} err="failed to get container status \"4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f\": rpc error: code = NotFound desc = could not find container \"4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f\": container with ID starting with 4932832ac27504c50d76c48145330208cb467d3ff5a9d9eea322b49afb3a6b5f not found: ID does not exist" Jul 20 16:15:47.793947 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.793924 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d7857bfc8-5l6mb"] Jul 20 16:15:47.798515 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:47.798493 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d7857bfc8-5l6mb"] Jul 20 16:15:48.326323 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:48.326289 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" path="/var/lib/kubelet/pods/c4085b90-2f60-4834-90f2-6a0b9bd5d980/volumes" Jul 20 16:15:54.644914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:15:54.644880 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xswbf" Jul 20 16:16:04.812049 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:04.811982 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d48dcdb85-9j2kh" podUID="e1e85e25-b114-4867-9360-f7326d96adc2" containerName="console" containerID="cri-o://ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795" gracePeriod=15 Jul 20 16:16:05.049932 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.049907 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:16:05.166490 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166399 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64hsd\" (UniqueName: \"kubernetes.io/projected/e1e85e25-b114-4867-9360-f7326d96adc2-kube-api-access-64hsd\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.166490 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166446 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-oauth-serving-cert\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.166490 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166473 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-console-config\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.166796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166504 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-oauth-config\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.166796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166609 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-serving-cert\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.166796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166724 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-trusted-ca-bundle\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.166796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166780 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-service-ca\") pod \"e1e85e25-b114-4867-9360-f7326d96adc2\" (UID: \"e1e85e25-b114-4867-9360-f7326d96adc2\") " Jul 20 16:16:05.167000 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166907 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:16:05.167000 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.166964 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-console-config" (OuterVolumeSpecName: "console-config") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:16:05.167135 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.167118 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-oauth-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.167184 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.167125 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:16:05.167184 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.167142 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-console-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.167261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.167202 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-service-ca" (OuterVolumeSpecName: "service-ca") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:16:05.168799 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.168774 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e85e25-b114-4867-9360-f7326d96adc2-kube-api-access-64hsd" (OuterVolumeSpecName: "kube-api-access-64hsd") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "kube-api-access-64hsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:16:05.169106 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.169080 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:16:05.169168 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.169129 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e1e85e25-b114-4867-9360-f7326d96adc2" (UID: "e1e85e25-b114-4867-9360-f7326d96adc2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:16:05.267759 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.267724 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64hsd\" (UniqueName: \"kubernetes.io/projected/e1e85e25-b114-4867-9360-f7326d96adc2-kube-api-access-64hsd\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.267759 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.267752 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-oauth-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.267759 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.267761 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1e85e25-b114-4867-9360-f7326d96adc2-console-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.267759 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.267769 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-trusted-ca-bundle\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.268016 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.267780 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e85e25-b114-4867-9360-f7326d96adc2-service-ca\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:16:05.842513 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.842475 2582 generic.go:356] "Generic (PLEG): container finished" podID="e1e85e25-b114-4867-9360-f7326d96adc2" containerID="ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795" exitCode=0 Jul 20 16:16:05.842972 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.842530 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d48dcdb85-9j2kh" Jul 20 16:16:05.842972 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.842565 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d48dcdb85-9j2kh" event={"ID":"e1e85e25-b114-4867-9360-f7326d96adc2","Type":"ContainerDied","Data":"ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795"} Jul 20 16:16:05.842972 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.842632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d48dcdb85-9j2kh" event={"ID":"e1e85e25-b114-4867-9360-f7326d96adc2","Type":"ContainerDied","Data":"e8f516cf5626ff1a8e9e4d7635b1ed7b56ca111adba683cd63f8b0844edbe0ba"} Jul 20 16:16:05.842972 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.842655 2582 scope.go:117] "RemoveContainer" containerID="ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795" Jul 20 16:16:05.851589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.851573 2582 scope.go:117] "RemoveContainer" containerID="ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795" Jul 20 16:16:05.851901 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:16:05.851884 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795\": container with ID starting with ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795 not found: ID does not exist" containerID="ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795" Jul 20 16:16:05.851964 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.851910 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795"} err="failed to get container status \"ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795\": rpc error: code = NotFound desc = could not find container \"ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795\": container with ID starting with ff093f0baa316a9f7b37a180d168ed35812f01b4df8a443f4d79a22f18aaf795 not found: ID does not exist" Jul 20 16:16:05.864576 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.864548 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d48dcdb85-9j2kh"] Jul 20 16:16:05.867631 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:05.867591 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d48dcdb85-9j2kh"] Jul 20 16:16:06.326319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:06.326284 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1e85e25-b114-4867-9360-f7326d96adc2" path="/var/lib/kubelet/pods/e1e85e25-b114-4867-9360-f7326d96adc2/volumes" Jul 20 16:16:29.432157 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:29.432114 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:16:29.451122 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:29.451098 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:16:29.947291 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:29.947264 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:16:53.573664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.573607 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5887766486-vjwpk"] Jul 20 16:16:53.574269 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.574247 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" containerName="console" Jul 20 16:16:53.574354 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.574273 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" containerName="console" Jul 20 16:16:53.574354 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.574288 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e85e25-b114-4867-9360-f7326d96adc2" containerName="console" Jul 20 16:16:53.574354 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.574296 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e85e25-b114-4867-9360-f7326d96adc2" containerName="console" Jul 20 16:16:53.574487 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.574446 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e85e25-b114-4867-9360-f7326d96adc2" containerName="console" Jul 20 16:16:53.574487 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.574462 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4085b90-2f60-4834-90f2-6a0b9bd5d980" containerName="console" Jul 20 16:16:53.578266 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.578248 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.595049 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.595021 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5887766486-vjwpk"] Jul 20 16:16:53.655059 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655032 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-oauth-serving-cert\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.655161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655073 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-trusted-ca-bundle\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.655161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-config\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.655161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-service-ca\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.655161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655156 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-oauth-config\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.655316 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655176 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kqx\" (UniqueName: \"kubernetes.io/projected/3e7014c2-3279-47dc-a9a3-ba3a273ce432-kube-api-access-22kqx\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.655316 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.655224 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-serving-cert\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756324 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-oauth-serving-cert\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756334 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-trusted-ca-bundle\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-config\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-service-ca\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756509 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-oauth-config\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756542 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756522 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kqx\" (UniqueName: \"kubernetes.io/projected/3e7014c2-3279-47dc-a9a3-ba3a273ce432-kube-api-access-22kqx\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.756662 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.756574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-serving-cert\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.757120 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.757095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-service-ca\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.757219 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.757196 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-oauth-serving-cert\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.757290 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.757196 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-config\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.757290 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.757203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-trusted-ca-bundle\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.759019 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.759001 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-serving-cert\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.759094 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.759045 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-oauth-config\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.764536 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.764507 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kqx\" (UniqueName: \"kubernetes.io/projected/3e7014c2-3279-47dc-a9a3-ba3a273ce432-kube-api-access-22kqx\") pod \"console-5887766486-vjwpk\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:53.887320 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:53.887273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:16:54.014832 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:54.014759 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5887766486-vjwpk"] Jul 20 16:16:54.017031 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:16:54.017000 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7014c2_3279_47dc_a9a3_ba3a273ce432.slice/crio-12bca4e45522f411ff7213c3d6da09152149f8a136340165f1eb5504aae506d8 WatchSource:0}: Error finding container 12bca4e45522f411ff7213c3d6da09152149f8a136340165f1eb5504aae506d8: Status 404 returned error can't find the container with id 12bca4e45522f411ff7213c3d6da09152149f8a136340165f1eb5504aae506d8 Jul 20 16:16:54.022234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:54.022204 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5887766486-vjwpk" event={"ID":"3e7014c2-3279-47dc-a9a3-ba3a273ce432","Type":"ContainerStarted","Data":"12bca4e45522f411ff7213c3d6da09152149f8a136340165f1eb5504aae506d8"} Jul 20 16:16:55.027068 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:16:55.027035 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5887766486-vjwpk" event={"ID":"3e7014c2-3279-47dc-a9a3-ba3a273ce432","Type":"ContainerStarted","Data":"18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb"} Jul 20 16:17:03.887547 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:03.887457 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:17:03.888127 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:03.887582 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:17:03.891872 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:03.891843 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:17:03.911686 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:03.911644 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5887766486-vjwpk" podStartSLOduration=10.911610879 podStartE2EDuration="10.911610879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:16:55.043213698 +0000 UTC m=+163.248359440" watchObservedRunningTime="2026-07-20 16:17:03.911610879 +0000 UTC m=+172.116756621" Jul 20 16:17:04.069844 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.069823 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:17:04.116241 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.116219 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb97fb888-ltwkd"] Jul 20 16:17:04.187780 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.187745 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77ccf9b4d4-s8fcq"] Jul 20 16:17:04.190979 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.190961 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.204584 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.204562 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77ccf9b4d4-s8fcq"] Jul 20 16:17:04.248172 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248136 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-oauth-config\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.248384 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248214 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-config\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.248384 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248293 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ldgh\" (UniqueName: \"kubernetes.io/projected/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-kube-api-access-4ldgh\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.248384 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248343 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-oauth-serving-cert\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.248519 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-service-ca\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.248519 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248440 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-trusted-ca-bundle\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.248519 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.248473 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-serving-cert\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349209 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349166 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldgh\" (UniqueName: \"kubernetes.io/projected/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-kube-api-access-4ldgh\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-oauth-serving-cert\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349269 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-service-ca\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-trusted-ca-bundle\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-serving-cert\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349655 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-oauth-config\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.349655 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.349604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-config\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.350076 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.350047 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-oauth-serving-cert\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.350188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.350060 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-service-ca\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.350246 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.350203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-config\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.350717 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.350696 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-trusted-ca-bundle\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.351938 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.351921 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-serving-cert\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.351993 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.351967 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-oauth-config\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.356902 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.356882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ldgh\" (UniqueName: \"kubernetes.io/projected/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-kube-api-access-4ldgh\") pod \"console-77ccf9b4d4-s8fcq\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.500083 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.499987 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:04.627165 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:04.627086 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77ccf9b4d4-s8fcq"] Jul 20 16:17:04.629887 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:17:04.629863 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ba48ef_f2c6_4392_a75b_7fd7140b4c85.slice/crio-d47dee1c7fca010e5d21dbd911bf3142d42c48eab03cba1090571dbdcaffa75f WatchSource:0}: Error finding container d47dee1c7fca010e5d21dbd911bf3142d42c48eab03cba1090571dbdcaffa75f: Status 404 returned error can't find the container with id d47dee1c7fca010e5d21dbd911bf3142d42c48eab03cba1090571dbdcaffa75f Jul 20 16:17:05.071776 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:05.071739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ccf9b4d4-s8fcq" event={"ID":"43ba48ef-f2c6-4392-a75b-7fd7140b4c85","Type":"ContainerStarted","Data":"c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678"} Jul 20 16:17:05.072141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:05.071796 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ccf9b4d4-s8fcq" event={"ID":"43ba48ef-f2c6-4392-a75b-7fd7140b4c85","Type":"ContainerStarted","Data":"d47dee1c7fca010e5d21dbd911bf3142d42c48eab03cba1090571dbdcaffa75f"} Jul 20 16:17:05.090060 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:05.090014 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77ccf9b4d4-s8fcq" podStartSLOduration=1.089994072 podStartE2EDuration="1.089994072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:17:05.089392957 +0000 UTC m=+173.294538700" watchObservedRunningTime="2026-07-20 16:17:05.089994072 +0000 UTC m=+173.295139816" Jul 20 16:17:14.500574 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:14.500541 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:14.501035 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:14.500592 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:14.506110 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:14.506086 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:15.120597 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:15.120552 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:17:15.170878 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:15.170851 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5887766486-vjwpk"] Jul 20 16:17:29.136398 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.136336 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fb97fb888-ltwkd" podUID="80507aae-8478-4d34-8daf-6feed87c6d63" containerName="console" containerID="cri-o://c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec" gracePeriod=15 Jul 20 16:17:29.379630 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.379594 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:17:29.443416 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443391 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-serving-cert\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.443524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443423 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-oauth-config\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.443524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443451 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-oauth-serving-cert\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.443524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443502 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-service-ca\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.443524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443517 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-console-config\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.443741 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443568 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-trusted-ca-bundle\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.443741 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443610 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22gsb\" (UniqueName: \"kubernetes.io/projected/80507aae-8478-4d34-8daf-6feed87c6d63-kube-api-access-22gsb\") pod \"80507aae-8478-4d34-8daf-6feed87c6d63\" (UID: \"80507aae-8478-4d34-8daf-6feed87c6d63\") " Jul 20 16:17:29.444144 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.443976 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:29.444144 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.444025 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-service-ca" (OuterVolumeSpecName: "service-ca") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:29.444144 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.444108 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-console-config" (OuterVolumeSpecName: "console-config") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:29.444386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.444363 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:29.445507 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.445482 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:17:29.445853 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.445833 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:17:29.445950 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.445896 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80507aae-8478-4d34-8daf-6feed87c6d63-kube-api-access-22gsb" (OuterVolumeSpecName: "kube-api-access-22gsb") pod "80507aae-8478-4d34-8daf-6feed87c6d63" (UID: "80507aae-8478-4d34-8daf-6feed87c6d63"). InnerVolumeSpecName "kube-api-access-22gsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:17:29.544793 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544775 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-trusted-ca-bundle\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:29.544793 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544794 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22gsb\" (UniqueName: \"kubernetes.io/projected/80507aae-8478-4d34-8daf-6feed87c6d63-kube-api-access-22gsb\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:29.544917 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544803 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:29.544917 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544813 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80507aae-8478-4d34-8daf-6feed87c6d63-console-oauth-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:29.544917 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544821 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-oauth-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:29.544917 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544831 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-service-ca\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:29.544917 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:29.544838 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80507aae-8478-4d34-8daf-6feed87c6d63-console-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:30.172070 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.172033 2582 generic.go:356] "Generic (PLEG): container finished" podID="80507aae-8478-4d34-8daf-6feed87c6d63" containerID="c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec" exitCode=0 Jul 20 16:17:30.172498 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.172093 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb97fb888-ltwkd" Jul 20 16:17:30.172498 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.172122 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb97fb888-ltwkd" event={"ID":"80507aae-8478-4d34-8daf-6feed87c6d63","Type":"ContainerDied","Data":"c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec"} Jul 20 16:17:30.172498 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.172214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb97fb888-ltwkd" event={"ID":"80507aae-8478-4d34-8daf-6feed87c6d63","Type":"ContainerDied","Data":"cbef3fbf36144e7f9630501c220c6df892380186b0b6a7991d08ef730ac8d6af"} Jul 20 16:17:30.172498 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.172243 2582 scope.go:117] "RemoveContainer" containerID="c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec" Jul 20 16:17:30.180342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.180326 2582 scope.go:117] "RemoveContainer" containerID="c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec" Jul 20 16:17:30.180584 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:17:30.180565 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec\": container with ID starting with c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec not found: ID does not exist" containerID="c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec" Jul 20 16:17:30.180680 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.180591 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec"} err="failed to get container status \"c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec\": rpc error: code = NotFound desc = could not find container \"c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec\": container with ID starting with c0cf1bdcb74d90510755b5d4e20943a2d5b63d66929c6af0321af2eca2397aec not found: ID does not exist" Jul 20 16:17:30.193030 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.193012 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb97fb888-ltwkd"] Jul 20 16:17:30.195878 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.195858 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fb97fb888-ltwkd"] Jul 20 16:17:30.326082 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:30.326057 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80507aae-8478-4d34-8daf-6feed87c6d63" path="/var/lib/kubelet/pods/80507aae-8478-4d34-8daf-6feed87c6d63/volumes" Jul 20 16:17:40.190666 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.190606 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5887766486-vjwpk" podUID="3e7014c2-3279-47dc-a9a3-ba3a273ce432" containerName="console" containerID="cri-o://18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb" gracePeriod=15 Jul 20 16:17:40.435220 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.435200 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:17:40.525395 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525318 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-config\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525395 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525371 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-oauth-config\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525395 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525392 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-serving-cert\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525424 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-oauth-serving-cert\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525453 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-trusted-ca-bundle\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525664 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525563 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22kqx\" (UniqueName: \"kubernetes.io/projected/3e7014c2-3279-47dc-a9a3-ba3a273ce432-kube-api-access-22kqx\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525826 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525708 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-service-ca\") pod \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\" (UID: \"3e7014c2-3279-47dc-a9a3-ba3a273ce432\") " Jul 20 16:17:40.525826 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525810 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-config" (OuterVolumeSpecName: "console-config") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:40.525935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525852 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:40.526037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.525920 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:40.526098 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.526077 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:40.526148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.526099 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-oauth-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:40.526148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.526103 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:17:40.526148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.526115 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-trusted-ca-bundle\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:40.527325 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.527302 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:17:40.527500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.527480 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:17:40.527589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.527536 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7014c2-3279-47dc-a9a3-ba3a273ce432-kube-api-access-22kqx" (OuterVolumeSpecName: "kube-api-access-22kqx") pod "3e7014c2-3279-47dc-a9a3-ba3a273ce432" (UID: "3e7014c2-3279-47dc-a9a3-ba3a273ce432"). InnerVolumeSpecName "kube-api-access-22kqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:17:40.626849 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.626826 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7014c2-3279-47dc-a9a3-ba3a273ce432-service-ca\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:40.626849 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.626848 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-oauth-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:40.626968 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.626857 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7014c2-3279-47dc-a9a3-ba3a273ce432-console-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:40.626968 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:40.626866 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22kqx\" (UniqueName: \"kubernetes.io/projected/3e7014c2-3279-47dc-a9a3-ba3a273ce432-kube-api-access-22kqx\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:17:41.213586 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.213552 2582 generic.go:356] "Generic (PLEG): container finished" podID="3e7014c2-3279-47dc-a9a3-ba3a273ce432" containerID="18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb" exitCode=0 Jul 20 16:17:41.213960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.213608 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5887766486-vjwpk" Jul 20 16:17:41.213960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.213610 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5887766486-vjwpk" event={"ID":"3e7014c2-3279-47dc-a9a3-ba3a273ce432","Type":"ContainerDied","Data":"18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb"} Jul 20 16:17:41.213960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.213690 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5887766486-vjwpk" event={"ID":"3e7014c2-3279-47dc-a9a3-ba3a273ce432","Type":"ContainerDied","Data":"12bca4e45522f411ff7213c3d6da09152149f8a136340165f1eb5504aae506d8"} Jul 20 16:17:41.213960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.213711 2582 scope.go:117] "RemoveContainer" containerID="18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb" Jul 20 16:17:41.223406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.223382 2582 scope.go:117] "RemoveContainer" containerID="18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb" Jul 20 16:17:41.224108 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:17:41.224086 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb\": container with ID starting with 18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb not found: ID does not exist" containerID="18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb" Jul 20 16:17:41.224222 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.224112 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb"} err="failed to get container status \"18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb\": rpc error: code = NotFound desc = could not find container \"18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb\": container with ID starting with 18c496167131cbc251e6c31cbb7f18755bf8af8effce40a76f6cfc6b8f8644fb not found: ID does not exist" Jul 20 16:17:41.235406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.235383 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5887766486-vjwpk"] Jul 20 16:17:41.238970 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:41.238945 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5887766486-vjwpk"] Jul 20 16:17:42.326230 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:17:42.326196 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e7014c2-3279-47dc-a9a3-ba3a273ce432" path="/var/lib/kubelet/pods/3e7014c2-3279-47dc-a9a3-ba3a273ce432/volumes" Jul 20 16:19:12.180445 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.180417 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 16:19:12.266172 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266145 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-92bvd"] Jul 20 16:19:12.266442 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266430 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80507aae-8478-4d34-8daf-6feed87c6d63" containerName="console" Jul 20 16:19:12.266486 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266443 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="80507aae-8478-4d34-8daf-6feed87c6d63" containerName="console" Jul 20 16:19:12.266486 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266464 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7014c2-3279-47dc-a9a3-ba3a273ce432" containerName="console" Jul 20 16:19:12.266486 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266469 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7014c2-3279-47dc-a9a3-ba3a273ce432" containerName="console" Jul 20 16:19:12.266575 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266515 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7014c2-3279-47dc-a9a3-ba3a273ce432" containerName="console" Jul 20 16:19:12.266575 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.266523 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="80507aae-8478-4d34-8daf-6feed87c6d63" containerName="console" Jul 20 16:19:12.269288 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.269270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.271734 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.271701 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-n2vrv\"" Jul 20 16:19:12.272672 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.272653 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 16:19:12.272733 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.272718 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 16:19:12.280701 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.280681 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-92bvd"] Jul 20 16:19:12.404615 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.404455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7fbj\" (UniqueName: \"kubernetes.io/projected/fdb5df75-07c2-479d-ad1c-6ac16a62affd-kube-api-access-s7fbj\") pod \"cert-manager-cainjector-559cdd8c87-92bvd\" (UID: \"fdb5df75-07c2-479d-ad1c-6ac16a62affd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.404615 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.404498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdb5df75-07c2-479d-ad1c-6ac16a62affd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-92bvd\" (UID: \"fdb5df75-07c2-479d-ad1c-6ac16a62affd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.505319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.505235 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7fbj\" (UniqueName: \"kubernetes.io/projected/fdb5df75-07c2-479d-ad1c-6ac16a62affd-kube-api-access-s7fbj\") pod \"cert-manager-cainjector-559cdd8c87-92bvd\" (UID: \"fdb5df75-07c2-479d-ad1c-6ac16a62affd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.505319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.505273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdb5df75-07c2-479d-ad1c-6ac16a62affd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-92bvd\" (UID: \"fdb5df75-07c2-479d-ad1c-6ac16a62affd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.513800 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.513767 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdb5df75-07c2-479d-ad1c-6ac16a62affd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-92bvd\" (UID: \"fdb5df75-07c2-479d-ad1c-6ac16a62affd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.513934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.513871 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7fbj\" (UniqueName: \"kubernetes.io/projected/fdb5df75-07c2-479d-ad1c-6ac16a62affd-kube-api-access-s7fbj\") pod \"cert-manager-cainjector-559cdd8c87-92bvd\" (UID: \"fdb5df75-07c2-479d-ad1c-6ac16a62affd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.593639 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.593584 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" Jul 20 16:19:12.712009 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:12.711970 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-92bvd"] Jul 20 16:19:12.714235 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:19:12.714208 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdb5df75_07c2_479d_ad1c_6ac16a62affd.slice/crio-9bd8112fab214945b10ab841b5e1c5aa8d3106d31831064fcf736afffa9f60f8 WatchSource:0}: Error finding container 9bd8112fab214945b10ab841b5e1c5aa8d3106d31831064fcf736afffa9f60f8: Status 404 returned error can't find the container with id 9bd8112fab214945b10ab841b5e1c5aa8d3106d31831064fcf736afffa9f60f8 Jul 20 16:19:13.542104 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:13.542064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" event={"ID":"fdb5df75-07c2-479d-ad1c-6ac16a62affd","Type":"ContainerStarted","Data":"9bd8112fab214945b10ab841b5e1c5aa8d3106d31831064fcf736afffa9f60f8"} Jul 20 16:19:16.558038 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:16.557997 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" event={"ID":"fdb5df75-07c2-479d-ad1c-6ac16a62affd","Type":"ContainerStarted","Data":"fa93cfc623bfc8524904cae51120242f0d07bb4ed532aa541b032e96d303d5db"} Jul 20 16:19:16.572587 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:16.572540 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-92bvd" podStartSLOduration=1.6701940720000001 podStartE2EDuration="4.572526256s" totalImagesPullingTime="2.902332184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:19:16.571900381 +0000 UTC m=+304.777046124" watchObservedRunningTime="2026-07-20 16:19:16.572526256 +0000 UTC m=+304.777671998" Jul 20 16:19:41.856599 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.856564 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv"] Jul 20 16:19:41.859867 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.859852 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:41.863034 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.863008 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 16:19:41.863168 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.863053 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 16:19:41.863436 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.863422 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 16:19:41.863727 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.863713 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 16:19:41.873289 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.873273 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xzk42\"" Jul 20 16:19:41.892380 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.892357 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv"] Jul 20 16:19:41.912234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.912209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9528ffd8-aac4-40e8-b261-35a585bbf31d-manifests\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:41.912340 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.912243 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5qt5\" (UniqueName: \"kubernetes.io/projected/9528ffd8-aac4-40e8-b261-35a585bbf31d-kube-api-access-h5qt5\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:41.912340 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.912266 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9528ffd8-aac4-40e8-b261-35a585bbf31d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:41.912340 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.912328 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9528ffd8-aac4-40e8-b261-35a585bbf31d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:41.912474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:41.912390 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9528ffd8-aac4-40e8-b261-35a585bbf31d-tmp\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.013124 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.013092 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qt5\" (UniqueName: \"kubernetes.io/projected/9528ffd8-aac4-40e8-b261-35a585bbf31d-kube-api-access-h5qt5\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.013124 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.013123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9528ffd8-aac4-40e8-b261-35a585bbf31d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.013301 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.013141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9528ffd8-aac4-40e8-b261-35a585bbf31d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.013301 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.013166 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9528ffd8-aac4-40e8-b261-35a585bbf31d-tmp\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.013301 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.013224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9528ffd8-aac4-40e8-b261-35a585bbf31d-manifests\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.013680 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.013651 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9528ffd8-aac4-40e8-b261-35a585bbf31d-manifests\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.015476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.015457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9528ffd8-aac4-40e8-b261-35a585bbf31d-tmp\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.015596 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.015579 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9528ffd8-aac4-40e8-b261-35a585bbf31d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.015714 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.015695 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9528ffd8-aac4-40e8-b261-35a585bbf31d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.027275 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.027250 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5qt5\" (UniqueName: \"kubernetes.io/projected/9528ffd8-aac4-40e8-b261-35a585bbf31d-kube-api-access-h5qt5\") pod \"opendatahub-operator-controller-manager-6bbb748554-9kcjv\" (UID: \"9528ffd8-aac4-40e8-b261-35a585bbf31d\") " pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.170411 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.170357 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:42.302284 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.302255 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv"] Jul 20 16:19:42.306415 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:19:42.306381 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9528ffd8_aac4_40e8_b261_35a585bbf31d.slice/crio-74b45e4cf3892f9a0a4277e0ebb55ada0207f9bad837e5179b6a402a494fb0fa WatchSource:0}: Error finding container 74b45e4cf3892f9a0a4277e0ebb55ada0207f9bad837e5179b6a402a494fb0fa: Status 404 returned error can't find the container with id 74b45e4cf3892f9a0a4277e0ebb55ada0207f9bad837e5179b6a402a494fb0fa Jul 20 16:19:42.308253 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.308238 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:19:42.655860 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:42.655832 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" event={"ID":"9528ffd8-aac4-40e8-b261-35a585bbf31d","Type":"ContainerStarted","Data":"74b45e4cf3892f9a0a4277e0ebb55ada0207f9bad837e5179b6a402a494fb0fa"} Jul 20 16:19:45.670766 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:45.670677 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" event={"ID":"9528ffd8-aac4-40e8-b261-35a585bbf31d","Type":"ContainerStarted","Data":"18665890f23ed4ffcc1a38243e94136a4ed2b27590e34a37a890914d0a3eaa94"} Jul 20 16:19:46.676078 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:46.676045 2582 generic.go:356] "Generic (PLEG): container finished" podID="9528ffd8-aac4-40e8-b261-35a585bbf31d" containerID="18665890f23ed4ffcc1a38243e94136a4ed2b27590e34a37a890914d0a3eaa94" exitCode=0 Jul 20 16:19:46.676429 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:46.676142 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" event={"ID":"9528ffd8-aac4-40e8-b261-35a585bbf31d","Type":"ContainerDied","Data":"18665890f23ed4ffcc1a38243e94136a4ed2b27590e34a37a890914d0a3eaa94"} Jul 20 16:19:47.681975 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:47.681939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" event={"ID":"9528ffd8-aac4-40e8-b261-35a585bbf31d","Type":"ContainerStarted","Data":"b144d9565d91e8ac7996fc7db4b6070b4b31eb5a1256b3c06196562c4137a05b"} Jul 20 16:19:47.682369 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:47.682066 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:47.704819 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:47.704771 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" podStartSLOduration=2.164591568 podStartE2EDuration="6.70475652s" totalImagesPullingTime="3.525458368s" totalInitContainerRuntime="1.014706584s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:19:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:19:47.703134541 +0000 UTC m=+335.908280284" watchObservedRunningTime="2026-07-20 16:19:47.70475652 +0000 UTC m=+335.909902262" Jul 20 16:19:58.687777 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:58.687745 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bbb748554-9kcjv" Jul 20 16:19:59.720462 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.720424 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr"] Jul 20 16:19:59.730147 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.730125 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.732475 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.732449 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr"] Jul 20 16:19:59.734312 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.734290 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 16:19:59.735238 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.735221 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 16:19:59.735335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.735223 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:19:59.735335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.735224 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w4fl7\"" Jul 20 16:19:59.735854 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.735836 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 16:19:59.735943 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.735894 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 16:19:59.847981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.847942 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2jtb\" (UniqueName: \"kubernetes.io/projected/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-kube-api-access-m2jtb\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.847981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.847983 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-metrics-cert\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.848203 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.848008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-manager-config\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.848203 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.848064 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-cert\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.948954 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.948925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2jtb\" (UniqueName: \"kubernetes.io/projected/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-kube-api-access-m2jtb\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.948954 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.948959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-metrics-cert\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.949150 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.948980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-manager-config\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.949150 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.949016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-cert\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.949730 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.949710 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-manager-config\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.951224 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.951199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-metrics-cert\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.951329 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.951253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-cert\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:19:59.956162 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:19:59.956141 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2jtb\" (UniqueName: \"kubernetes.io/projected/5b4216d7-9f37-4525-9399-d1e6fcf0c98f-kube-api-access-m2jtb\") pod \"lws-controller-manager-7797c4479d-f9kmr\" (UID: \"5b4216d7-9f37-4525-9399-d1e6fcf0c98f\") " pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:20:00.040356 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:00.040280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:20:00.164051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:00.164027 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr"] Jul 20 16:20:00.165902 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:20:00.165874 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4216d7_9f37_4525_9399_d1e6fcf0c98f.slice/crio-e7b059b375648f72ed09ecb3220e7b360f1523b3a344cd393427e7baf3689895 WatchSource:0}: Error finding container e7b059b375648f72ed09ecb3220e7b360f1523b3a344cd393427e7baf3689895: Status 404 returned error can't find the container with id e7b059b375648f72ed09ecb3220e7b360f1523b3a344cd393427e7baf3689895 Jul 20 16:20:00.732403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:00.732367 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" event={"ID":"5b4216d7-9f37-4525-9399-d1e6fcf0c98f","Type":"ContainerStarted","Data":"e7b059b375648f72ed09ecb3220e7b360f1523b3a344cd393427e7baf3689895"} Jul 20 16:20:02.611930 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.611884 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw"] Jul 20 16:20:02.615330 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.615300 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.618596 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.618544 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 16:20:02.618596 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.618557 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 16:20:02.618596 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.618577 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 16:20:02.618880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.618858 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 16:20:02.619657 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.619639 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xh5g8\"" Jul 20 16:20:02.628463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.628444 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw"] Jul 20 16:20:02.673655 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.673629 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4456216b-ebab-4f9b-b312-9bd03055ab3a-tls-certs\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.673752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.673664 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdmqz\" (UniqueName: \"kubernetes.io/projected/4456216b-ebab-4f9b-b312-9bd03055ab3a-kube-api-access-vdmqz\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.673752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.673694 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4456216b-ebab-4f9b-b312-9bd03055ab3a-tmp\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.744176 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.744147 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" event={"ID":"5b4216d7-9f37-4525-9399-d1e6fcf0c98f","Type":"ContainerStarted","Data":"849758f5ff3895bebc948c9ad15ff0b310519f2a9b2dd1f302d79c69c1216ed0"} Jul 20 16:20:02.744312 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.744269 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:20:02.775101 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.775075 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4456216b-ebab-4f9b-b312-9bd03055ab3a-tls-certs\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.775101 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.775104 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdmqz\" (UniqueName: \"kubernetes.io/projected/4456216b-ebab-4f9b-b312-9bd03055ab3a-kube-api-access-vdmqz\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.775272 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.775128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4456216b-ebab-4f9b-b312-9bd03055ab3a-tmp\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.777387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.777369 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4456216b-ebab-4f9b-b312-9bd03055ab3a-tmp\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.777596 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.777577 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4456216b-ebab-4f9b-b312-9bd03055ab3a-tls-certs\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.779936 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.779896 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" podStartSLOduration=1.313479902 podStartE2EDuration="3.779882294s" totalImagesPullingTime="2.466402392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:20:02.779549929 +0000 UTC m=+350.984695673" watchObservedRunningTime="2026-07-20 16:20:02.779882294 +0000 UTC m=+350.985028036" Jul 20 16:20:02.790744 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.790725 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdmqz\" (UniqueName: \"kubernetes.io/projected/4456216b-ebab-4f9b-b312-9bd03055ab3a-kube-api-access-vdmqz\") pod \"kube-auth-proxy-5b4f98b7bc-4krfw\" (UID: \"4456216b-ebab-4f9b-b312-9bd03055ab3a\") " pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:02.926328 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:02.926241 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" Jul 20 16:20:03.062238 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:03.062210 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw"] Jul 20 16:20:03.064657 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:20:03.064611 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4456216b_ebab_4f9b_b312_9bd03055ab3a.slice/crio-d30e50dbd9003618516ec138af4ab196826eae955ffe7e902a1721631671d58f WatchSource:0}: Error finding container d30e50dbd9003618516ec138af4ab196826eae955ffe7e902a1721631671d58f: Status 404 returned error can't find the container with id d30e50dbd9003618516ec138af4ab196826eae955ffe7e902a1721631671d58f Jul 20 16:20:03.751269 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:03.751231 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" event={"ID":"4456216b-ebab-4f9b-b312-9bd03055ab3a","Type":"ContainerStarted","Data":"d30e50dbd9003618516ec138af4ab196826eae955ffe7e902a1721631671d58f"} Jul 20 16:20:06.767901 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:06.767865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" event={"ID":"4456216b-ebab-4f9b-b312-9bd03055ab3a","Type":"ContainerStarted","Data":"55655a4f6af0dd2c19db542888734850c7b149bfee38f528884c0bef3e7fca0b"} Jul 20 16:20:06.785364 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:06.785312 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b4f98b7bc-4krfw" podStartSLOduration=1.768188135 podStartE2EDuration="4.785297279s" totalImagesPullingTime="3.017109144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:20:06.78311536 +0000 UTC m=+354.988261102" watchObservedRunningTime="2026-07-20 16:20:06.785297279 +0000 UTC m=+354.990443020" Jul 20 16:20:10.551654 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.551605 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7"] Jul 20 16:20:10.555130 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.555111 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.558062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.558042 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 16:20:10.558234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.558211 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-66qk9\"" Jul 20 16:20:10.574793 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.574761 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7"] Jul 20 16:20:10.644125 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644223 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644223 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644206 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644295 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644295 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644357 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644303 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644357 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2c4q\" (UniqueName: \"kubernetes.io/projected/fbbc242c-5106-41e2-b37d-1964f0d9c0da-kube-api-access-d2c4q\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644357 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644354 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.644448 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.644378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745254 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745222 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745265 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745369 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745410 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745724 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745471 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745724 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745498 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2c4q\" (UniqueName: \"kubernetes.io/projected/fbbc242c-5106-41e2-b37d-1964f0d9c0da-kube-api-access-d2c4q\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745724 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745977 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745951 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.745977 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.745967 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.746137 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.746055 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.746485 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.746461 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.746563 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.746528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.748091 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.748056 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.748197 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.748145 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.753841 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.753812 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbc242c-5106-41e2-b37d-1964f0d9c0da-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.754200 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.754184 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2c4q\" (UniqueName: \"kubernetes.io/projected/fbbc242c-5106-41e2-b37d-1964f0d9c0da-kube-api-access-d2c4q\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46slp7\" (UID: \"fbbc242c-5106-41e2-b37d-1964f0d9c0da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:10.865612 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:10.865539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:20:11.032165 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:11.032136 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7"] Jul 20 16:20:11.034610 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:20:11.034579 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbc242c_5106_41e2_b37d_1964f0d9c0da.slice/crio-86f4e41893d6b2085efc30b7357918f82a8d5691dcf06b4e3704c05b7ce87039 WatchSource:0}: Error finding container 86f4e41893d6b2085efc30b7357918f82a8d5691dcf06b4e3704c05b7ce87039: Status 404 returned error can't find the container with id 86f4e41893d6b2085efc30b7357918f82a8d5691dcf06b4e3704c05b7ce87039 Jul 20 16:20:11.789185 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:11.789145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" event={"ID":"fbbc242c-5106-41e2-b37d-1964f0d9c0da","Type":"ContainerStarted","Data":"86f4e41893d6b2085efc30b7357918f82a8d5691dcf06b4e3704c05b7ce87039"} Jul 20 16:20:13.753568 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:20:13.753528 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7797c4479d-f9kmr" Jul 20 16:21:00.806307 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:00.806267 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:21:00.806721 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:00.806358 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:21:00.806721 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:00.806404 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:21:01.001998 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:01.001959 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" event={"ID":"fbbc242c-5106-41e2-b37d-1964f0d9c0da","Type":"ContainerStarted","Data":"f6bd4e7f4e0d9c0384c1baec54c21d06c4b01023eac77cc2b7f78d5f8f68ec72"} Jul 20 16:21:01.025892 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:01.025835 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" podStartSLOduration=1.256576879 podStartE2EDuration="51.025817612s" totalImagesPullingTime="49.769240733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:21:01.023087192 +0000 UTC m=+409.228232948" watchObservedRunningTime="2026-07-20 16:21:01.025817612 +0000 UTC m=+409.230963354" Jul 20 16:21:01.866713 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:01.866679 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:21:01.871273 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:01.871249 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:21:02.006215 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:02.006189 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:21:02.006961 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:02.006944 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46slp7" Jul 20 16:21:50.640421 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.640332 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk"] Jul 20 16:21:50.647713 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.647687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.650992 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.650967 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 16:21:50.651146 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.650974 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 16:21:50.651146 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.650974 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nc4th\"" Jul 20 16:21:50.651589 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.651573 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 16:21:50.652252 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.652235 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 16:21:50.664779 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.664743 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk"] Jul 20 16:21:50.722982 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.722936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd778\" (UniqueName: \"kubernetes.io/projected/3c00db93-d944-4b47-aaf8-abf47985b9f7-kube-api-access-zd778\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.723167 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.723036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c00db93-d944-4b47-aaf8-abf47985b9f7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.723167 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.723097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3c00db93-d944-4b47-aaf8-abf47985b9f7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.823507 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.823465 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3c00db93-d944-4b47-aaf8-abf47985b9f7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.823507 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.823503 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd778\" (UniqueName: \"kubernetes.io/projected/3c00db93-d944-4b47-aaf8-abf47985b9f7-kube-api-access-zd778\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.823736 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.823561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c00db93-d944-4b47-aaf8-abf47985b9f7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.823736 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:21:50.823707 2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 20 16:21:50.823804 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:21:50.823774 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c00db93-d944-4b47-aaf8-abf47985b9f7-plugin-serving-cert podName:3c00db93-d944-4b47-aaf8-abf47985b9f7 nodeName:}" failed. No retries permitted until 2026-07-20 16:21:51.323753226 +0000 UTC m=+459.528898946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3c00db93-d944-4b47-aaf8-abf47985b9f7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-czhkk" (UID: "3c00db93-d944-4b47-aaf8-abf47985b9f7") : secret "plugin-serving-cert" not found Jul 20 16:21:50.824175 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.824149 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3c00db93-d944-4b47-aaf8-abf47985b9f7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:50.834975 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:50.834938 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd778\" (UniqueName: \"kubernetes.io/projected/3c00db93-d944-4b47-aaf8-abf47985b9f7-kube-api-access-zd778\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:51.326592 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:51.326557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c00db93-d944-4b47-aaf8-abf47985b9f7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:51.329144 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:51.329118 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c00db93-d944-4b47-aaf8-abf47985b9f7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-czhkk\" (UID: \"3c00db93-d944-4b47-aaf8-abf47985b9f7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:51.556706 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:51.556673 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" Jul 20 16:21:51.683835 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:51.683809 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk"] Jul 20 16:21:51.685745 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:21:51.685718 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c00db93_d944_4b47_aaf8_abf47985b9f7.slice/crio-bf10ae99a215f381167a4b9e5d217f19cfbf74fb0c19db77885f612f7b22c572 WatchSource:0}: Error finding container bf10ae99a215f381167a4b9e5d217f19cfbf74fb0c19db77885f612f7b22c572: Status 404 returned error can't find the container with id bf10ae99a215f381167a4b9e5d217f19cfbf74fb0c19db77885f612f7b22c572 Jul 20 16:21:52.225429 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:52.225397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" event={"ID":"3c00db93-d944-4b47-aaf8-abf47985b9f7","Type":"ContainerStarted","Data":"bf10ae99a215f381167a4b9e5d217f19cfbf74fb0c19db77885f612f7b22c572"} Jul 20 16:21:57.192547 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.192514 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75b5ffb47c-whthc"] Jul 20 16:21:57.207992 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.207962 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b5ffb47c-whthc"] Jul 20 16:21:57.208163 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.208091 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.277658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-serving-cert\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.277658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-oauth-serving-cert\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.277859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277761 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-oauth-config\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.277859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277822 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-trusted-ca-bundle\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.277859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277857 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpnpv\" (UniqueName: \"kubernetes.io/projected/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-kube-api-access-jpnpv\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.278008 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277878 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-service-ca\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.278008 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.277953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-config\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379182 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-oauth-config\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-trusted-ca-bundle\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpnpv\" (UniqueName: \"kubernetes.io/projected/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-kube-api-access-jpnpv\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-service-ca\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379301 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-config\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-serving-cert\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.379601 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.379361 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-oauth-serving-cert\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.380115 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.380086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-oauth-serving-cert\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.380243 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.380220 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-trusted-ca-bundle\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.380306 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.380247 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-service-ca\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.380526 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.380506 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-config\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.381970 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.381947 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-oauth-config\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.382273 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.382251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-console-serving-cert\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.388775 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.388755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpnpv\" (UniqueName: \"kubernetes.io/projected/3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965-kube-api-access-jpnpv\") pod \"console-75b5ffb47c-whthc\" (UID: \"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965\") " pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:57.520082 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:57.519998 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:21:58.635851 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:58.635824 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b5ffb47c-whthc"] Jul 20 16:21:58.638744 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:21:58.638716 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3668c7b0_ffc6_4ef1_93b0_f4d5d6d76965.slice/crio-8e6464e13f17a219a43da80ba405b7898652d580cc32e44592da04bdfff44533 WatchSource:0}: Error finding container 8e6464e13f17a219a43da80ba405b7898652d580cc32e44592da04bdfff44533: Status 404 returned error can't find the container with id 8e6464e13f17a219a43da80ba405b7898652d580cc32e44592da04bdfff44533 Jul 20 16:21:59.261829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:59.261792 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b5ffb47c-whthc" event={"ID":"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965","Type":"ContainerStarted","Data":"9174538781d71c2a6fb4bb04dbea5d6f8c2cfaa87d06a2db3777579f44037181"} Jul 20 16:21:59.262022 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:59.261846 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b5ffb47c-whthc" event={"ID":"3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965","Type":"ContainerStarted","Data":"8e6464e13f17a219a43da80ba405b7898652d580cc32e44592da04bdfff44533"} Jul 20 16:21:59.282152 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:21:59.282104 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75b5ffb47c-whthc" podStartSLOduration=2.282088323 podStartE2EDuration="2.282088323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:21:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:21:59.280078388 +0000 UTC m=+467.485224135" watchObservedRunningTime="2026-07-20 16:21:59.282088323 +0000 UTC m=+467.487234064" Jul 20 16:22:01.668605 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.668569 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz"] Jul 20 16:22:01.678321 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.678301 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:01.681359 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.681338 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n542q\"" Jul 20 16:22:01.688052 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.688031 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz"] Jul 20 16:22:01.720179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.720143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b06d840-2ed1-434a-8da9-23acce2ccf3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" (UID: \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:01.720179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.720189 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nthl\" (UniqueName: \"kubernetes.io/projected/8b06d840-2ed1-434a-8da9-23acce2ccf3a-kube-api-access-6nthl\") pod \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" (UID: \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:01.736193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.736166 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz"] Jul 20 16:22:01.736493 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:01.736461 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-6nthl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" podUID="8b06d840-2ed1-434a-8da9-23acce2ccf3a" Jul 20 16:22:01.752207 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.752180 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz"] Jul 20 16:22:01.772914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.772887 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2"] Jul 20 16:22:01.776630 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.776598 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:01.793709 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.793684 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2"] Jul 20 16:22:01.821380 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.821350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjd7\" (UniqueName: \"kubernetes.io/projected/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-kube-api-access-kgjd7\") pod \"kuadrant-operator-controller-manager-86878c769f-6gkf2\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:01.821534 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.821506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6gkf2\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:01.821608 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.821578 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b06d840-2ed1-434a-8da9-23acce2ccf3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" (UID: \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:01.821670 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.821633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nthl\" (UniqueName: \"kubernetes.io/projected/8b06d840-2ed1-434a-8da9-23acce2ccf3a-kube-api-access-6nthl\") pod \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" (UID: \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:01.822069 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.822031 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b06d840-2ed1-434a-8da9-23acce2ccf3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" (UID: \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:01.826780 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:01.826747 2582 projected.go:196] Error preparing data for projected volume kube-api-access-6nthl for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-ntwpz" not found Jul 20 16:22:01.826871 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:01.826815 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b06d840-2ed1-434a-8da9-23acce2ccf3a-kube-api-access-6nthl podName:8b06d840-2ed1-434a-8da9-23acce2ccf3a nodeName:}" failed. No retries permitted until 2026-07-20 16:22:02.326799433 +0000 UTC m=+470.531945152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6nthl" (UniqueName: "kubernetes.io/projected/8b06d840-2ed1-434a-8da9-23acce2ccf3a-kube-api-access-6nthl") pod "kuadrant-operator-controller-manager-86878c769f-ntwpz" (UID: "8b06d840-2ed1-434a-8da9-23acce2ccf3a") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-ntwpz" not found Jul 20 16:22:01.922873 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.922794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6gkf2\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:01.923023 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.922911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjd7\" (UniqueName: \"kubernetes.io/projected/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-kube-api-access-kgjd7\") pod \"kuadrant-operator-controller-manager-86878c769f-6gkf2\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:01.923271 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.923243 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6gkf2\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:01.932061 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:01.932035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjd7\" (UniqueName: \"kubernetes.io/projected/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-kube-api-access-kgjd7\") pod \"kuadrant-operator-controller-manager-86878c769f-6gkf2\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:02.089928 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.089899 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:02.274796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.274753 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:02.277246 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:02.277209 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" is forbidden: User \"system:node:ip-10-0-131-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-211.ec2.internal' and this object" podUID="8b06d840-2ed1-434a-8da9-23acce2ccf3a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:02.288562 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.288540 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:02.290827 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:02.290793 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" is forbidden: User \"system:node:ip-10-0-131-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-211.ec2.internal' and this object" podUID="8b06d840-2ed1-434a-8da9-23acce2ccf3a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:02.326244 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:02.326213 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" is forbidden: User \"system:node:ip-10-0-131-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-211.ec2.internal' and this object" podUID="8b06d840-2ed1-434a-8da9-23acce2ccf3a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:02.326446 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.326421 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b06d840-2ed1-434a-8da9-23acce2ccf3a-extensions-socket-volume\") pod \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\" (UID: \"8b06d840-2ed1-434a-8da9-23acce2ccf3a\") " Jul 20 16:22:02.326658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.326632 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b06d840-2ed1-434a-8da9-23acce2ccf3a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8b06d840-2ed1-434a-8da9-23acce2ccf3a" (UID: "8b06d840-2ed1-434a-8da9-23acce2ccf3a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:22:02.326658 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.326653 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nthl\" (UniqueName: \"kubernetes.io/projected/8b06d840-2ed1-434a-8da9-23acce2ccf3a-kube-api-access-6nthl\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:02.327535 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.327516 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b06d840-2ed1-434a-8da9-23acce2ccf3a" path="/var/lib/kubelet/pods/8b06d840-2ed1-434a-8da9-23acce2ccf3a/volumes" Jul 20 16:22:02.428009 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:02.427976 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b06d840-2ed1-434a-8da9-23acce2ccf3a-extensions-socket-volume\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:03.280204 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:03.280166 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:03.285454 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:03.285418 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ntwpz\" is forbidden: User \"system:node:ip-10-0-131-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-211.ec2.internal' and this object" podUID="8b06d840-2ed1-434a-8da9-23acce2ccf3a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntwpz" Jul 20 16:22:07.520918 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:07.520887 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:22:07.521328 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:07.520950 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:22:07.526138 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:07.526113 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:22:08.309360 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:08.309334 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75b5ffb47c-whthc" Jul 20 16:22:08.375245 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:08.375205 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77ccf9b4d4-s8fcq"] Jul 20 16:22:14.897870 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:14.897841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2"] Jul 20 16:22:14.898522 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:22:14.898489 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3700f4fd_cfcd_4c24_81a9_e00c18d02d63.slice/crio-0d48cb6141382a659cb93cd05990132e17a42a0bedd7a9816ba1590c104e83f3 WatchSource:0}: Error finding container 0d48cb6141382a659cb93cd05990132e17a42a0bedd7a9816ba1590c104e83f3: Status 404 returned error can't find the container with id 0d48cb6141382a659cb93cd05990132e17a42a0bedd7a9816ba1590c104e83f3 Jul 20 16:22:15.344194 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:15.344149 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" event={"ID":"3c00db93-d944-4b47-aaf8-abf47985b9f7","Type":"ContainerStarted","Data":"ae711f17e1a5be10e14db9c6aa6989adb7b2be88d8ef81e9b1602e6c1902e624"} Jul 20 16:22:15.345437 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:15.345412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" event={"ID":"3700f4fd-cfcd-4c24-81a9-e00c18d02d63","Type":"ContainerStarted","Data":"0d48cb6141382a659cb93cd05990132e17a42a0bedd7a9816ba1590c104e83f3"} Jul 20 16:22:15.360579 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:15.360537 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-czhkk" podStartSLOduration=1.8846887909999999 podStartE2EDuration="25.360523185s" totalImagesPullingTime="23.475834394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:22:15.359643612 +0000 UTC m=+483.564789355" watchObservedRunningTime="2026-07-20 16:22:15.360523185 +0000 UTC m=+483.565668927" Jul 20 16:22:22.380234 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:22.380151 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" event={"ID":"3700f4fd-cfcd-4c24-81a9-e00c18d02d63","Type":"ContainerStarted","Data":"a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67"} Jul 20 16:22:22.380590 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:22.380262 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:22.399808 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:22.399767 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" podStartSLOduration=14.277457199 podStartE2EDuration="21.399748461s" totalImagesPullingTime="7.122291262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:22:22.397564159 +0000 UTC m=+490.602709904" watchObservedRunningTime="2026-07-20 16:22:22.399748461 +0000 UTC m=+490.604894203" Jul 20 16:22:33.387111 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.387077 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:22:33.406079 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.406017 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77ccf9b4d4-s8fcq" podUID="43ba48ef-f2c6-4392-a75b-7fd7140b4c85" containerName="console" containerID="cri-o://c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678" gracePeriod=15 Jul 20 16:22:33.655066 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.655042 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:22:33.724226 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724197 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ldgh\" (UniqueName: \"kubernetes.io/projected/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-kube-api-access-4ldgh\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724226 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724231 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-oauth-serving-cert\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724251 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-trusted-ca-bundle\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724291 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-oauth-config\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724313 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-service-ca\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724342 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-serving-cert\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724416 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-config\") pod \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\" (UID: \"43ba48ef-f2c6-4392-a75b-7fd7140b4c85\") " Jul 20 16:22:33.724757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724713 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:22:33.724844 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.724788 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-service-ca" (OuterVolumeSpecName: "service-ca") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:22:33.725046 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.725023 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-config" (OuterVolumeSpecName: "console-config") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:22:33.725121 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.725041 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:22:33.726386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.726357 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:22:33.726518 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.726483 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:22:33.726798 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.726765 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-kube-api-access-4ldgh" (OuterVolumeSpecName: "kube-api-access-4ldgh") pod "43ba48ef-f2c6-4392-a75b-7fd7140b4c85" (UID: "43ba48ef-f2c6-4392-a75b-7fd7140b4c85"). InnerVolumeSpecName "kube-api-access-4ldgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:22:33.825177 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825145 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ldgh\" (UniqueName: \"kubernetes.io/projected/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-kube-api-access-4ldgh\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:33.825177 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825174 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-oauth-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:33.825177 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825184 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-trusted-ca-bundle\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:33.825412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825193 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-oauth-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:33.825412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825202 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-service-ca\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:33.825412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825211 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-serving-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:33.825412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:33.825219 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43ba48ef-f2c6-4392-a75b-7fd7140b4c85-console-config\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:22:34.432657 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.432600 2582 generic.go:356] "Generic (PLEG): container finished" podID="43ba48ef-f2c6-4392-a75b-7fd7140b4c85" containerID="c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678" exitCode=0 Jul 20 16:22:34.433062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.432668 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ccf9b4d4-s8fcq" Jul 20 16:22:34.433062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.432686 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ccf9b4d4-s8fcq" event={"ID":"43ba48ef-f2c6-4392-a75b-7fd7140b4c85","Type":"ContainerDied","Data":"c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678"} Jul 20 16:22:34.433062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.432739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ccf9b4d4-s8fcq" event={"ID":"43ba48ef-f2c6-4392-a75b-7fd7140b4c85","Type":"ContainerDied","Data":"d47dee1c7fca010e5d21dbd911bf3142d42c48eab03cba1090571dbdcaffa75f"} Jul 20 16:22:34.433062 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.432759 2582 scope.go:117] "RemoveContainer" containerID="c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678" Jul 20 16:22:34.440804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.440783 2582 scope.go:117] "RemoveContainer" containerID="c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678" Jul 20 16:22:34.441086 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:22:34.441067 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678\": container with ID starting with c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678 not found: ID does not exist" containerID="c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678" Jul 20 16:22:34.441137 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.441105 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678"} err="failed to get container status \"c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678\": rpc error: code = NotFound desc = could not find container \"c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678\": container with ID starting with c55f3c26fbfec55af55927313bf112e3ce3e2fac98481abbf3a0984595e24678 not found: ID does not exist" Jul 20 16:22:34.455515 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.455485 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77ccf9b4d4-s8fcq"] Jul 20 16:22:34.459466 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:34.459442 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77ccf9b4d4-s8fcq"] Jul 20 16:22:36.327723 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:36.327686 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43ba48ef-f2c6-4392-a75b-7fd7140b4c85" path="/var/lib/kubelet/pods/43ba48ef-f2c6-4392-a75b-7fd7140b4c85/volumes" Jul 20 16:22:55.108143 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.108111 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-jzzxw"] Jul 20 16:22:55.108611 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.108433 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43ba48ef-f2c6-4392-a75b-7fd7140b4c85" containerName="console" Jul 20 16:22:55.108611 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.108443 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="43ba48ef-f2c6-4392-a75b-7fd7140b4c85" containerName="console" Jul 20 16:22:55.114046 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.112887 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="43ba48ef-f2c6-4392-a75b-7fd7140b4c85" containerName="console" Jul 20 16:22:55.176069 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.176043 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jzzxw"] Jul 20 16:22:55.176190 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.176123 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:22:55.178766 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.178742 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ggqpn\"" Jul 20 16:22:55.262957 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.262935 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-sx8bf"] Jul 20 16:22:55.266127 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.266111 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:22:55.273520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.273502 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sx8bf"] Jul 20 16:22:55.297875 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.297853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cstcb\" (UniqueName: \"kubernetes.io/projected/776a58eb-c284-44aa-abfe-e95aec158205-kube-api-access-cstcb\") pod \"authorino-58cd955675-jzzxw\" (UID: \"776a58eb-c284-44aa-abfe-e95aec158205\") " pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:22:55.398948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.398884 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdcgt\" (UniqueName: \"kubernetes.io/projected/27824887-ce3d-4d7b-8a53-693c6d01572b-kube-api-access-vdcgt\") pod \"authorino-75d556bd77-sx8bf\" (UID: \"27824887-ce3d-4d7b-8a53-693c6d01572b\") " pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:22:55.399058 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.398963 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cstcb\" (UniqueName: \"kubernetes.io/projected/776a58eb-c284-44aa-abfe-e95aec158205-kube-api-access-cstcb\") pod \"authorino-58cd955675-jzzxw\" (UID: \"776a58eb-c284-44aa-abfe-e95aec158205\") " pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:22:55.407073 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.407050 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cstcb\" (UniqueName: \"kubernetes.io/projected/776a58eb-c284-44aa-abfe-e95aec158205-kube-api-access-cstcb\") pod \"authorino-58cd955675-jzzxw\" (UID: \"776a58eb-c284-44aa-abfe-e95aec158205\") " pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:22:55.484949 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.484928 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:22:55.500188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.500164 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdcgt\" (UniqueName: \"kubernetes.io/projected/27824887-ce3d-4d7b-8a53-693c6d01572b-kube-api-access-vdcgt\") pod \"authorino-75d556bd77-sx8bf\" (UID: \"27824887-ce3d-4d7b-8a53-693c6d01572b\") " pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:22:55.508549 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.508529 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdcgt\" (UniqueName: \"kubernetes.io/projected/27824887-ce3d-4d7b-8a53-693c6d01572b-kube-api-access-vdcgt\") pod \"authorino-75d556bd77-sx8bf\" (UID: \"27824887-ce3d-4d7b-8a53-693c6d01572b\") " pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:22:55.574934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.574907 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:22:55.614332 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.614303 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jzzxw"] Jul 20 16:22:55.615373 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:22:55.615346 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod776a58eb_c284_44aa_abfe_e95aec158205.slice/crio-de62f3d80aaec32979c5951562834b04b03ad693093d0946fa69b8a61f3d708b WatchSource:0}: Error finding container de62f3d80aaec32979c5951562834b04b03ad693093d0946fa69b8a61f3d708b: Status 404 returned error can't find the container with id de62f3d80aaec32979c5951562834b04b03ad693093d0946fa69b8a61f3d708b Jul 20 16:22:55.702977 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:55.702923 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sx8bf"] Jul 20 16:22:55.707725 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:22:55.706966 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27824887_ce3d_4d7b_8a53_693c6d01572b.slice/crio-c5b5e512f2fe168e9b76169092ebf1fb4c8df5ffe438a4f6f52459186e411ab0 WatchSource:0}: Error finding container c5b5e512f2fe168e9b76169092ebf1fb4c8df5ffe438a4f6f52459186e411ab0: Status 404 returned error can't find the container with id c5b5e512f2fe168e9b76169092ebf1fb4c8df5ffe438a4f6f52459186e411ab0 Jul 20 16:22:56.530923 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:56.530867 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sx8bf" event={"ID":"27824887-ce3d-4d7b-8a53-693c6d01572b","Type":"ContainerStarted","Data":"c5b5e512f2fe168e9b76169092ebf1fb4c8df5ffe438a4f6f52459186e411ab0"} Jul 20 16:22:56.532667 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:56.532609 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jzzxw" event={"ID":"776a58eb-c284-44aa-abfe-e95aec158205","Type":"ContainerStarted","Data":"de62f3d80aaec32979c5951562834b04b03ad693093d0946fa69b8a61f3d708b"} Jul 20 16:22:59.549960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:59.549869 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jzzxw" event={"ID":"776a58eb-c284-44aa-abfe-e95aec158205","Type":"ContainerStarted","Data":"9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c"} Jul 20 16:22:59.551315 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:59.551287 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sx8bf" event={"ID":"27824887-ce3d-4d7b-8a53-693c6d01572b","Type":"ContainerStarted","Data":"d2562a51bce6771e8980735f09d0d5c2f5a5812b71b01b3b981c265773bccd4a"} Jul 20 16:22:59.569081 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:59.568987 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-jzzxw" podStartSLOduration=0.985613064 podStartE2EDuration="4.568970156s" totalImagesPullingTime="3.583357092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:22:59.567441062 +0000 UTC m=+527.772586804" watchObservedRunningTime="2026-07-20 16:22:59.568970156 +0000 UTC m=+527.774115903" Jul 20 16:22:59.627356 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:59.627306 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-sx8bf" podStartSLOduration=1.129744523 podStartE2EDuration="4.627284422s" totalImagesPullingTime="3.497539899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:22:59.595319396 +0000 UTC m=+527.800465157" watchObservedRunningTime="2026-07-20 16:22:59.627284422 +0000 UTC m=+527.832430164" Jul 20 16:22:59.628642 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:22:59.628602 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jzzxw"] Jul 20 16:23:01.560271 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:01.560228 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-jzzxw" podUID="776a58eb-c284-44aa-abfe-e95aec158205" containerName="authorino" containerID="cri-o://9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c" gracePeriod=30 Jul 20 16:23:01.797294 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:01.797269 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:23:01.961886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:01.961856 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cstcb\" (UniqueName: \"kubernetes.io/projected/776a58eb-c284-44aa-abfe-e95aec158205-kube-api-access-cstcb\") pod \"776a58eb-c284-44aa-abfe-e95aec158205\" (UID: \"776a58eb-c284-44aa-abfe-e95aec158205\") " Jul 20 16:23:01.963962 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:01.963938 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/776a58eb-c284-44aa-abfe-e95aec158205-kube-api-access-cstcb" (OuterVolumeSpecName: "kube-api-access-cstcb") pod "776a58eb-c284-44aa-abfe-e95aec158205" (UID: "776a58eb-c284-44aa-abfe-e95aec158205"). InnerVolumeSpecName "kube-api-access-cstcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:23:02.063279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.063252 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cstcb\" (UniqueName: \"kubernetes.io/projected/776a58eb-c284-44aa-abfe-e95aec158205-kube-api-access-cstcb\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:23:02.565458 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.565426 2582 generic.go:356] "Generic (PLEG): container finished" podID="776a58eb-c284-44aa-abfe-e95aec158205" containerID="9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c" exitCode=0 Jul 20 16:23:02.565833 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.565464 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jzzxw" Jul 20 16:23:02.565833 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.565502 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jzzxw" event={"ID":"776a58eb-c284-44aa-abfe-e95aec158205","Type":"ContainerDied","Data":"9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c"} Jul 20 16:23:02.565833 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.565540 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jzzxw" event={"ID":"776a58eb-c284-44aa-abfe-e95aec158205","Type":"ContainerDied","Data":"de62f3d80aaec32979c5951562834b04b03ad693093d0946fa69b8a61f3d708b"} Jul 20 16:23:02.565833 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.565560 2582 scope.go:117] "RemoveContainer" containerID="9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c" Jul 20 16:23:02.573579 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.573559 2582 scope.go:117] "RemoveContainer" containerID="9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c" Jul 20 16:23:02.573851 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:23:02.573834 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c\": container with ID starting with 9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c not found: ID does not exist" containerID="9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c" Jul 20 16:23:02.573906 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.573859 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c"} err="failed to get container status \"9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c\": rpc error: code = NotFound desc = could not find container \"9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c\": container with ID starting with 9336cae67766b4b017f493a0e96687baf981fe1003b992f14fb8b08c9558c46c not found: ID does not exist" Jul 20 16:23:02.583498 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.583480 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jzzxw"] Jul 20 16:23:02.585908 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:02.585864 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-jzzxw"] Jul 20 16:23:04.327672 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:04.327582 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="776a58eb-c284-44aa-abfe-e95aec158205" path="/var/lib/kubelet/pods/776a58eb-c284-44aa-abfe-e95aec158205/volumes" Jul 20 16:23:24.219899 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.219864 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-689b6dc6f8-7wg4v"] Jul 20 16:23:24.220282 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.220222 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="776a58eb-c284-44aa-abfe-e95aec158205" containerName="authorino" Jul 20 16:23:24.220282 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.220233 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="776a58eb-c284-44aa-abfe-e95aec158205" containerName="authorino" Jul 20 16:23:24.220356 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.220293 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="776a58eb-c284-44aa-abfe-e95aec158205" containerName="authorino" Jul 20 16:23:24.223217 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.223196 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:24.231157 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.231134 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-689b6dc6f8-7wg4v"] Jul 20 16:23:24.239924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.239899 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v95x\" (UniqueName: \"kubernetes.io/projected/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0-kube-api-access-5v95x\") pod \"authorino-689b6dc6f8-7wg4v\" (UID: \"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0\") " pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:24.340576 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.340545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v95x\" (UniqueName: \"kubernetes.io/projected/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0-kube-api-access-5v95x\") pod \"authorino-689b6dc6f8-7wg4v\" (UID: \"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0\") " pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:24.349511 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.349483 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v95x\" (UniqueName: \"kubernetes.io/projected/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0-kube-api-access-5v95x\") pod \"authorino-689b6dc6f8-7wg4v\" (UID: \"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0\") " pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:24.459485 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.459460 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-689b6dc6f8-7wg4v"] Jul 20 16:23:24.459683 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.459672 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:24.582642 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.582597 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-689b6dc6f8-7wg4v"] Jul 20 16:23:24.584785 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:23:24.584757 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcea5dca0_b69d_4ab0_9eb7_c7dc839d3cb0.slice/crio-b193311f82ce706d7ce5e7bfaf8bc61c7a987421deeca9fd900ed02c1a6321a2 WatchSource:0}: Error finding container b193311f82ce706d7ce5e7bfaf8bc61c7a987421deeca9fd900ed02c1a6321a2: Status 404 returned error can't find the container with id b193311f82ce706d7ce5e7bfaf8bc61c7a987421deeca9fd900ed02c1a6321a2 Jul 20 16:23:24.668491 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:24.668460 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" event={"ID":"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0","Type":"ContainerStarted","Data":"b193311f82ce706d7ce5e7bfaf8bc61c7a987421deeca9fd900ed02c1a6321a2"} Jul 20 16:23:25.674548 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:25.674517 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" event={"ID":"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0","Type":"ContainerStarted","Data":"7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4"} Jul 20 16:23:25.674971 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:25.674602 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" podUID="cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" containerName="authorino" containerID="cri-o://7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4" gracePeriod=30 Jul 20 16:23:25.690984 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:25.690943 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" podStartSLOduration=1.373782937 podStartE2EDuration="1.690927286s" totalImagesPullingTime="317.144349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:23:25.689297618 +0000 UTC m=+553.894443367" watchObservedRunningTime="2026-07-20 16:23:25.690927286 +0000 UTC m=+553.896073027" Jul 20 16:23:25.916537 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:25.916515 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:25.953117 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:25.953096 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v95x\" (UniqueName: \"kubernetes.io/projected/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0-kube-api-access-5v95x\") pod \"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0\" (UID: \"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0\") " Jul 20 16:23:25.954893 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:25.954873 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0-kube-api-access-5v95x" (OuterVolumeSpecName: "kube-api-access-5v95x") pod "cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" (UID: "cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0"). InnerVolumeSpecName "kube-api-access-5v95x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:23:26.053942 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.053917 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v95x\" (UniqueName: \"kubernetes.io/projected/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0-kube-api-access-5v95x\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:23:26.679989 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.679959 2582 generic.go:356] "Generic (PLEG): container finished" podID="cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" containerID="7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4" exitCode=0 Jul 20 16:23:26.680342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.679999 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" Jul 20 16:23:26.680342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.680042 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" event={"ID":"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0","Type":"ContainerDied","Data":"7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4"} Jul 20 16:23:26.680342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.680080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-689b6dc6f8-7wg4v" event={"ID":"cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0","Type":"ContainerDied","Data":"b193311f82ce706d7ce5e7bfaf8bc61c7a987421deeca9fd900ed02c1a6321a2"} Jul 20 16:23:26.680342 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.680097 2582 scope.go:117] "RemoveContainer" containerID="7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4" Jul 20 16:23:26.692412 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.692395 2582 scope.go:117] "RemoveContainer" containerID="7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4" Jul 20 16:23:26.692680 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:23:26.692654 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4\": container with ID starting with 7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4 not found: ID does not exist" containerID="7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4" Jul 20 16:23:26.692785 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.692687 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4"} err="failed to get container status \"7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4\": rpc error: code = NotFound desc = could not find container \"7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4\": container with ID starting with 7ed379f0023cae30037ec61e890b9b19434700285550d2f77e93b5603d5d7ac4 not found: ID does not exist" Jul 20 16:23:26.701353 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.701330 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-689b6dc6f8-7wg4v"] Jul 20 16:23:26.705156 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:26.705136 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-689b6dc6f8-7wg4v"] Jul 20 16:23:28.326975 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.326937 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" path="/var/lib/kubelet/pods/cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0/volumes" Jul 20 16:23:28.486944 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.486910 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sx8bf"] Jul 20 16:23:28.487122 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.487099 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-sx8bf" podUID="27824887-ce3d-4d7b-8a53-693c6d01572b" containerName="authorino" containerID="cri-o://d2562a51bce6771e8980735f09d0d5c2f5a5812b71b01b3b981c265773bccd4a" gracePeriod=30 Jul 20 16:23:28.691173 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.691139 2582 generic.go:356] "Generic (PLEG): container finished" podID="27824887-ce3d-4d7b-8a53-693c6d01572b" containerID="d2562a51bce6771e8980735f09d0d5c2f5a5812b71b01b3b981c265773bccd4a" exitCode=0 Jul 20 16:23:28.691316 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.691198 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sx8bf" event={"ID":"27824887-ce3d-4d7b-8a53-693c6d01572b","Type":"ContainerDied","Data":"d2562a51bce6771e8980735f09d0d5c2f5a5812b71b01b3b981c265773bccd4a"} Jul 20 16:23:28.727376 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.727354 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:23:28.773764 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.773739 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdcgt\" (UniqueName: \"kubernetes.io/projected/27824887-ce3d-4d7b-8a53-693c6d01572b-kube-api-access-vdcgt\") pod \"27824887-ce3d-4d7b-8a53-693c6d01572b\" (UID: \"27824887-ce3d-4d7b-8a53-693c6d01572b\") " Jul 20 16:23:28.775670 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.775635 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27824887-ce3d-4d7b-8a53-693c6d01572b-kube-api-access-vdcgt" (OuterVolumeSpecName: "kube-api-access-vdcgt") pod "27824887-ce3d-4d7b-8a53-693c6d01572b" (UID: "27824887-ce3d-4d7b-8a53-693c6d01572b"). InnerVolumeSpecName "kube-api-access-vdcgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:23:28.874920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:28.874869 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdcgt\" (UniqueName: \"kubernetes.io/projected/27824887-ce3d-4d7b-8a53-693c6d01572b-kube-api-access-vdcgt\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:23:29.696754 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:29.696719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sx8bf" event={"ID":"27824887-ce3d-4d7b-8a53-693c6d01572b","Type":"ContainerDied","Data":"c5b5e512f2fe168e9b76169092ebf1fb4c8df5ffe438a4f6f52459186e411ab0"} Jul 20 16:23:29.696754 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:29.696738 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sx8bf" Jul 20 16:23:29.697225 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:29.696771 2582 scope.go:117] "RemoveContainer" containerID="d2562a51bce6771e8980735f09d0d5c2f5a5812b71b01b3b981c265773bccd4a" Jul 20 16:23:29.724828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:29.724805 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sx8bf"] Jul 20 16:23:29.735477 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:29.735455 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sx8bf"] Jul 20 16:23:30.327038 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:30.327009 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27824887-ce3d-4d7b-8a53-693c6d01572b" path="/var/lib/kubelet/pods/27824887-ce3d-4d7b-8a53-693c6d01572b/volumes" Jul 20 16:23:33.605948 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.605918 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b54448d8-9hss4"] Jul 20 16:23:33.606367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.606250 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27824887-ce3d-4d7b-8a53-693c6d01572b" containerName="authorino" Jul 20 16:23:33.606367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.606260 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="27824887-ce3d-4d7b-8a53-693c6d01572b" containerName="authorino" Jul 20 16:23:33.606367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.606273 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" containerName="authorino" Jul 20 16:23:33.606367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.606279 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" containerName="authorino" Jul 20 16:23:33.606367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.606333 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea5dca0-b69d-4ab0-9eb7-c7dc839d3cb0" containerName="authorino" Jul 20 16:23:33.606367 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.606342 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="27824887-ce3d-4d7b-8a53-693c6d01572b" containerName="authorino" Jul 20 16:23:33.644225 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.644200 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b54448d8-9hss4"] Jul 20 16:23:33.644368 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.644302 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.649415 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.649393 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 16:23:33.649840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.649802 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 16:23:33.650525 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.650509 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8674p\"" Jul 20 16:23:33.716539 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.716514 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-webhook-cert\") pod \"maas-controller-67b54448d8-9hss4\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.716666 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.716577 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhkcw\" (UniqueName: \"kubernetes.io/projected/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-kube-api-access-bhkcw\") pod \"maas-controller-67b54448d8-9hss4\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.817407 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.817382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhkcw\" (UniqueName: \"kubernetes.io/projected/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-kube-api-access-bhkcw\") pod \"maas-controller-67b54448d8-9hss4\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.817522 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.817441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-webhook-cert\") pod \"maas-controller-67b54448d8-9hss4\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.819788 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.819770 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-webhook-cert\") pod \"maas-controller-67b54448d8-9hss4\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.826285 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.826263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhkcw\" (UniqueName: \"kubernetes.io/projected/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-kube-api-access-bhkcw\") pod \"maas-controller-67b54448d8-9hss4\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:33.953961 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:33.953943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:34.080742 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:34.080714 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b54448d8-9hss4"] Jul 20 16:23:34.085122 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:23:34.085093 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18768ff4_2a0f_4b3a_8b7d_130ae195ff01.slice/crio-c954d141c32ff6a8d2d578b532713f3792780c6c8a0beb20494b8f369a9b7f81 WatchSource:0}: Error finding container c954d141c32ff6a8d2d578b532713f3792780c6c8a0beb20494b8f369a9b7f81: Status 404 returned error can't find the container with id c954d141c32ff6a8d2d578b532713f3792780c6c8a0beb20494b8f369a9b7f81 Jul 20 16:23:34.722175 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:34.722136 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-9hss4" event={"ID":"18768ff4-2a0f-4b3a-8b7d-130ae195ff01","Type":"ContainerStarted","Data":"c954d141c32ff6a8d2d578b532713f3792780c6c8a0beb20494b8f369a9b7f81"} Jul 20 16:23:36.734742 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:36.734708 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-9hss4" event={"ID":"18768ff4-2a0f-4b3a-8b7d-130ae195ff01","Type":"ContainerStarted","Data":"a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb"} Jul 20 16:23:36.735142 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:36.734830 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:23:36.764520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:36.764473 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b54448d8-9hss4" podStartSLOduration=1.552518188 podStartE2EDuration="3.764459513s" totalImagesPullingTime="2.211941325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:23:36.763195854 +0000 UTC m=+564.968341595" watchObservedRunningTime="2026-07-20 16:23:36.764459513 +0000 UTC m=+564.969605254" Jul 20 16:23:47.742122 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:23:47.742093 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:24:09.463717 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.463675 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wkpz9"] Jul 20 16:24:09.467088 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.467069 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.469823 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.469802 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 20 16:24:09.469929 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.469802 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vwsw6\"" Jul 20 16:24:09.478559 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.478538 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wkpz9"] Jul 20 16:24:09.517794 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.517770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2t8\" (UniqueName: \"kubernetes.io/projected/bf527d47-1635-4c85-9b2a-2040d021bb18-kube-api-access-tx2t8\") pod \"payload-processing-5bf47cd988-wkpz9\" (UID: \"bf527d47-1635-4c85-9b2a-2040d021bb18\") " pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.517907 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.517886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf527d47-1635-4c85-9b2a-2040d021bb18-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wkpz9\" (UID: \"bf527d47-1635-4c85-9b2a-2040d021bb18\") " pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.619257 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.619225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf527d47-1635-4c85-9b2a-2040d021bb18-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wkpz9\" (UID: \"bf527d47-1635-4c85-9b2a-2040d021bb18\") " pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.619409 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.619301 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2t8\" (UniqueName: \"kubernetes.io/projected/bf527d47-1635-4c85-9b2a-2040d021bb18-kube-api-access-tx2t8\") pod \"payload-processing-5bf47cd988-wkpz9\" (UID: \"bf527d47-1635-4c85-9b2a-2040d021bb18\") " pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.620075 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.620056 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf527d47-1635-4c85-9b2a-2040d021bb18-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wkpz9\" (UID: \"bf527d47-1635-4c85-9b2a-2040d021bb18\") " pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.627489 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.627465 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2t8\" (UniqueName: \"kubernetes.io/projected/bf527d47-1635-4c85-9b2a-2040d021bb18-kube-api-access-tx2t8\") pod \"payload-processing-5bf47cd988-wkpz9\" (UID: \"bf527d47-1635-4c85-9b2a-2040d021bb18\") " pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.778097 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.778040 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:09.912599 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:09.912576 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wkpz9"] Jul 20 16:24:09.914966 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:24:09.914930 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf527d47_1635_4c85_9b2a_2040d021bb18.slice/crio-e440258b4419df51735e30d860b6447309859d609249d51fd9c683c1b6db5dff WatchSource:0}: Error finding container e440258b4419df51735e30d860b6447309859d609249d51fd9c683c1b6db5dff: Status 404 returned error can't find the container with id e440258b4419df51735e30d860b6447309859d609249d51fd9c683c1b6db5dff Jul 20 16:24:10.880349 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:10.880299 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" event={"ID":"bf527d47-1635-4c85-9b2a-2040d021bb18","Type":"ContainerStarted","Data":"e440258b4419df51735e30d860b6447309859d609249d51fd9c683c1b6db5dff"} Jul 20 16:24:11.886391 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:11.886355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" event={"ID":"bf527d47-1635-4c85-9b2a-2040d021bb18","Type":"ContainerStarted","Data":"2136a86c91aeebbd3d295b83321913dcb77b1729c540d20748265771a4c77c44"} Jul 20 16:24:11.887041 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:11.886490 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:24:11.903084 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:11.903031 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" podStartSLOduration=1.100746432 podStartE2EDuration="2.903017726s" totalImagesPullingTime="1.802271294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:24:11.901968349 +0000 UTC m=+600.107114091" watchObservedRunningTime="2026-07-20 16:24:11.903017726 +0000 UTC m=+600.108163467" Jul 20 16:24:22.892535 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:24:22.892500 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-wkpz9" Jul 20 16:25:00.811574 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.811541 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk"] Jul 20 16:25:00.814288 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.814270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.818109 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.818085 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 20 16:25:00.818218 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.818086 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 16:25:00.818218 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.818087 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mmwbr\"" Jul 20 16:25:00.818218 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.818098 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 16:25:00.824966 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.824944 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk"] Jul 20 16:25:00.848032 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.848005 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.848183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.848057 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zzb\" (UniqueName: \"kubernetes.io/projected/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-kube-api-access-58zzb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.848183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.848111 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.848183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.848133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.848287 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.848187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.848287 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.848234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.948934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.948892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949123 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.948991 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949123 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949043 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zzb\" (UniqueName: \"kubernetes.io/projected/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-kube-api-access-58zzb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949123 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949123 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949094 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949328 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949340 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949371 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.949524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.949506 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.951270 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.951251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.951441 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.951421 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:00.957235 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:00.957212 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zzb\" (UniqueName: \"kubernetes.io/projected/40dfa8ce-5eb2-4cf1-8fa9-254aa153058c-kube-api-access-58zzb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9t7mk\" (UID: \"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:01.125169 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:01.125084 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:01.254757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:01.254733 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk"] Jul 20 16:25:01.256867 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:25:01.256837 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dfa8ce_5eb2_4cf1_8fa9_254aa153058c.slice/crio-c62095c45d5f807aa64c734337ece5be691e9414cfa973ec2eeaa4d0de6ec79c WatchSource:0}: Error finding container c62095c45d5f807aa64c734337ece5be691e9414cfa973ec2eeaa4d0de6ec79c: Status 404 returned error can't find the container with id c62095c45d5f807aa64c734337ece5be691e9414cfa973ec2eeaa4d0de6ec79c Jul 20 16:25:01.258416 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:01.258397 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:25:02.102526 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:02.102493 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" event={"ID":"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c","Type":"ContainerStarted","Data":"c62095c45d5f807aa64c734337ece5be691e9414cfa973ec2eeaa4d0de6ec79c"} Jul 20 16:25:03.107804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:03.107771 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" event={"ID":"40dfa8ce-5eb2-4cf1-8fa9-254aa153058c","Type":"ContainerStarted","Data":"8a0b660e94c5234c033092fd8f5bd2f3044473417049c888ba158f1a56850e0b"} Jul 20 16:25:03.125586 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:03.125526 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" podStartSLOduration=1.9372512899999998 podStartE2EDuration="3.125509094s" totalImagesPullingTime="1.188257804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:25:03.124584119 +0000 UTC m=+651.329729861" watchObservedRunningTime="2026-07-20 16:25:03.125509094 +0000 UTC m=+651.330654836" Jul 20 16:25:11.126066 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:11.126028 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:11.126461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:11.126119 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:11.138531 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:11.138507 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:11.155048 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:11.155024 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9t7mk" Jul 20 16:25:21.392728 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.392692 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82"] Jul 20 16:25:21.395217 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.395199 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.397726 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.397707 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 20 16:25:21.405256 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.405233 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82"] Jul 20 16:25:21.537143 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.537108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.537143 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.537150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.537363 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.537200 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9de5a5-6f02-4577-9088-29c38f901d58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.537363 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.537233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.537363 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.537335 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhqm5\" (UniqueName: \"kubernetes.io/projected/3a9de5a5-6f02-4577-9088-29c38f901d58-kube-api-access-dhqm5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.537475 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.537394 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638022 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.637980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhqm5\" (UniqueName: \"kubernetes.io/projected/3a9de5a5-6f02-4577-9088-29c38f901d58-kube-api-access-dhqm5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638125 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9de5a5-6f02-4577-9088-29c38f901d58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638188 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638181 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638550 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638701 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638582 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.638701 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.638675 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.640473 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.640452 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9de5a5-6f02-4577-9088-29c38f901d58-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.640771 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.640750 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9de5a5-6f02-4577-9088-29c38f901d58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.645828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.645775 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhqm5\" (UniqueName: \"kubernetes.io/projected/3a9de5a5-6f02-4577-9088-29c38f901d58-kube-api-access-dhqm5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9xh82\" (UID: \"3a9de5a5-6f02-4577-9088-29c38f901d58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.704482 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.704456 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:21.837949 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:21.837919 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82"] Jul 20 16:25:21.839970 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:25:21.839937 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a9de5a5_6f02_4577_9088_29c38f901d58.slice/crio-4248ce6fcb105ca9261697515324e3a88237cdf5d0b4247323dadbf3b7a2a946 WatchSource:0}: Error finding container 4248ce6fcb105ca9261697515324e3a88237cdf5d0b4247323dadbf3b7a2a946: Status 404 returned error can't find the container with id 4248ce6fcb105ca9261697515324e3a88237cdf5d0b4247323dadbf3b7a2a946 Jul 20 16:25:22.197191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:22.197155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" event={"ID":"3a9de5a5-6f02-4577-9088-29c38f901d58","Type":"ContainerStarted","Data":"54717c3238aa044f67ec43e5c003e3a81f989d84b8c4640ec0ce9baed45277cc"} Jul 20 16:25:22.197191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:22.197200 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" event={"ID":"3a9de5a5-6f02-4577-9088-29c38f901d58","Type":"ContainerStarted","Data":"4248ce6fcb105ca9261697515324e3a88237cdf5d0b4247323dadbf3b7a2a946"} Jul 20 16:25:22.214991 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:22.214926 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" podStartSLOduration=0.987053359 podStartE2EDuration="1.214911978s" totalImagesPullingTime="227.858619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:25:22.213565968 +0000 UTC m=+670.418711710" watchObservedRunningTime="2026-07-20 16:25:22.214911978 +0000 UTC m=+670.420057721" Jul 20 16:25:31.705166 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:31.705127 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:31.705568 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:31.705183 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:31.717791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:31.717767 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:25:32.255445 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:25:32.255414 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9xh82" Jul 20 16:26:10.296468 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.296434 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f4658cbf5-6nbvj"] Jul 20 16:26:10.298991 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.298973 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.302744 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.302721 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 20 16:26:10.302744 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.302729 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ggqpn\"" Jul 20 16:26:10.308741 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.308718 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4658cbf5-6nbvj"] Jul 20 16:26:10.365830 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.365796 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88bfe08d-42c1-43af-9333-4a75539fb80e-tls-cert\") pod \"authorino-5f4658cbf5-6nbvj\" (UID: \"88bfe08d-42c1-43af-9333-4a75539fb80e\") " pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.365830 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.365833 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-849wf\" (UniqueName: \"kubernetes.io/projected/88bfe08d-42c1-43af-9333-4a75539fb80e-kube-api-access-849wf\") pod \"authorino-5f4658cbf5-6nbvj\" (UID: \"88bfe08d-42c1-43af-9333-4a75539fb80e\") " pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.466704 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.466673 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88bfe08d-42c1-43af-9333-4a75539fb80e-tls-cert\") pod \"authorino-5f4658cbf5-6nbvj\" (UID: \"88bfe08d-42c1-43af-9333-4a75539fb80e\") " pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.467024 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.466996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-849wf\" (UniqueName: \"kubernetes.io/projected/88bfe08d-42c1-43af-9333-4a75539fb80e-kube-api-access-849wf\") pod \"authorino-5f4658cbf5-6nbvj\" (UID: \"88bfe08d-42c1-43af-9333-4a75539fb80e\") " pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.476966 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.476682 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88bfe08d-42c1-43af-9333-4a75539fb80e-tls-cert\") pod \"authorino-5f4658cbf5-6nbvj\" (UID: \"88bfe08d-42c1-43af-9333-4a75539fb80e\") " pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.477962 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.477934 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-849wf\" (UniqueName: \"kubernetes.io/projected/88bfe08d-42c1-43af-9333-4a75539fb80e-kube-api-access-849wf\") pod \"authorino-5f4658cbf5-6nbvj\" (UID: \"88bfe08d-42c1-43af-9333-4a75539fb80e\") " pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.608828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.608746 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" Jul 20 16:26:10.744777 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:10.744749 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4658cbf5-6nbvj"] Jul 20 16:26:10.746588 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:26:10.746560 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88bfe08d_42c1_43af_9333_4a75539fb80e.slice/crio-5ea8951c5ad82272dfc9b5dff3138f3c4d7de9325900b6919a66a01a32117b94 WatchSource:0}: Error finding container 5ea8951c5ad82272dfc9b5dff3138f3c4d7de9325900b6919a66a01a32117b94: Status 404 returned error can't find the container with id 5ea8951c5ad82272dfc9b5dff3138f3c4d7de9325900b6919a66a01a32117b94 Jul 20 16:26:11.420978 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:11.420936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" event={"ID":"88bfe08d-42c1-43af-9333-4a75539fb80e","Type":"ContainerStarted","Data":"5e68817f2f4589b17d3e335855f1f222d39443f735c1a1dc57ab7dac740a7d83"} Jul 20 16:26:11.421385 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:11.420989 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" event={"ID":"88bfe08d-42c1-43af-9333-4a75539fb80e","Type":"ContainerStarted","Data":"5ea8951c5ad82272dfc9b5dff3138f3c4d7de9325900b6919a66a01a32117b94"} Jul 20 16:26:11.449972 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:11.449920 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5f4658cbf5-6nbvj" podStartSLOduration=0.990228837 podStartE2EDuration="1.449901411s" totalImagesPullingTime="459.672574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:26:11.449688676 +0000 UTC m=+719.654834452" watchObservedRunningTime="2026-07-20 16:26:11.449901411 +0000 UTC m=+719.655047152" Jul 20 16:26:56.993574 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:56.993530 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b54448d8-9hss4"] Jul 20 16:26:56.994156 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:56.993825 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b54448d8-9hss4" podUID="18768ff4-2a0f-4b3a-8b7d-130ae195ff01" containerName="manager" containerID="cri-o://a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb" gracePeriod=30 Jul 20 16:26:57.245159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.245095 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:26:57.395299 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.395269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhkcw\" (UniqueName: \"kubernetes.io/projected/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-kube-api-access-bhkcw\") pod \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " Jul 20 16:26:57.395299 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.395301 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-webhook-cert\") pod \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\" (UID: \"18768ff4-2a0f-4b3a-8b7d-130ae195ff01\") " Jul 20 16:26:57.397417 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.397390 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-kube-api-access-bhkcw" (OuterVolumeSpecName: "kube-api-access-bhkcw") pod "18768ff4-2a0f-4b3a-8b7d-130ae195ff01" (UID: "18768ff4-2a0f-4b3a-8b7d-130ae195ff01"). InnerVolumeSpecName "kube-api-access-bhkcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:26:57.397417 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.397390 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "18768ff4-2a0f-4b3a-8b7d-130ae195ff01" (UID: "18768ff4-2a0f-4b3a-8b7d-130ae195ff01"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:26:57.496033 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.495971 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhkcw\" (UniqueName: \"kubernetes.io/projected/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-kube-api-access-bhkcw\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:26:57.496033 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.495995 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18768ff4-2a0f-4b3a-8b7d-130ae195ff01-webhook-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:26:57.637042 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.637013 2582 generic.go:356] "Generic (PLEG): container finished" podID="18768ff4-2a0f-4b3a-8b7d-130ae195ff01" containerID="a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb" exitCode=0 Jul 20 16:26:57.637159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.637062 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-9hss4" event={"ID":"18768ff4-2a0f-4b3a-8b7d-130ae195ff01","Type":"ContainerDied","Data":"a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb"} Jul 20 16:26:57.637159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.637067 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-9hss4" Jul 20 16:26:57.637159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.637104 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-9hss4" event={"ID":"18768ff4-2a0f-4b3a-8b7d-130ae195ff01","Type":"ContainerDied","Data":"c954d141c32ff6a8d2d578b532713f3792780c6c8a0beb20494b8f369a9b7f81"} Jul 20 16:26:57.637159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.637125 2582 scope.go:117] "RemoveContainer" containerID="a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb" Jul 20 16:26:57.645863 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.645839 2582 scope.go:117] "RemoveContainer" containerID="a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb" Jul 20 16:26:57.646098 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:26:57.646080 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb\": container with ID starting with a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb not found: ID does not exist" containerID="a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb" Jul 20 16:26:57.646169 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.646105 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb"} err="failed to get container status \"a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb\": rpc error: code = NotFound desc = could not find container \"a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb\": container with ID starting with a78d611d41a65bd18afd9f036da3700909112484592d456cd5e0c73a10a357cb not found: ID does not exist" Jul 20 16:26:57.657490 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.657470 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b54448d8-9hss4"] Jul 20 16:26:57.663424 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:57.663405 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b54448d8-9hss4"] Jul 20 16:26:58.328193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:26:58.328160 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18768ff4-2a0f-4b3a-8b7d-130ae195ff01" path="/var/lib/kubelet/pods/18768ff4-2a0f-4b3a-8b7d-130ae195ff01/volumes" Jul 20 16:27:58.432248 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.432218 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b54448d8-xh8t9"] Jul 20 16:27:58.432722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.432574 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18768ff4-2a0f-4b3a-8b7d-130ae195ff01" containerName="manager" Jul 20 16:27:58.432722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.432587 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18768ff4-2a0f-4b3a-8b7d-130ae195ff01" containerName="manager" Jul 20 16:27:58.432722 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.432665 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18768ff4-2a0f-4b3a-8b7d-130ae195ff01" containerName="manager" Jul 20 16:27:58.434725 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.434711 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.437191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.437166 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 16:27:58.438279 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.438261 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 16:27:58.438414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.438330 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8674p\"" Jul 20 16:27:58.445527 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.445509 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b54448d8-xh8t9"] Jul 20 16:27:58.587737 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.587713 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d711f142-170e-4fff-80c7-6c000dae7fa7-webhook-cert\") pod \"maas-controller-67b54448d8-xh8t9\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.587857 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.587746 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j289x\" (UniqueName: \"kubernetes.io/projected/d711f142-170e-4fff-80c7-6c000dae7fa7-kube-api-access-j289x\") pod \"maas-controller-67b54448d8-xh8t9\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.688382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.688313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d711f142-170e-4fff-80c7-6c000dae7fa7-webhook-cert\") pod \"maas-controller-67b54448d8-xh8t9\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.688382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.688350 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j289x\" (UniqueName: \"kubernetes.io/projected/d711f142-170e-4fff-80c7-6c000dae7fa7-kube-api-access-j289x\") pod \"maas-controller-67b54448d8-xh8t9\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.690794 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.690774 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d711f142-170e-4fff-80c7-6c000dae7fa7-webhook-cert\") pod \"maas-controller-67b54448d8-xh8t9\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.696528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.696505 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j289x\" (UniqueName: \"kubernetes.io/projected/d711f142-170e-4fff-80c7-6c000dae7fa7-kube-api-access-j289x\") pod \"maas-controller-67b54448d8-xh8t9\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.745478 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.745454 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:58.869000 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.868977 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b54448d8-xh8t9"] Jul 20 16:27:58.871221 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:27:58.871195 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd711f142_170e_4fff_80c7_6c000dae7fa7.slice/crio-e05be417e32006363014fd1aaf83746b4afd196b784e6ae64fefb0d2ec601873 WatchSource:0}: Error finding container e05be417e32006363014fd1aaf83746b4afd196b784e6ae64fefb0d2ec601873: Status 404 returned error can't find the container with id e05be417e32006363014fd1aaf83746b4afd196b784e6ae64fefb0d2ec601873 Jul 20 16:27:58.910439 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:58.910415 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-xh8t9" event={"ID":"d711f142-170e-4fff-80c7-6c000dae7fa7","Type":"ContainerStarted","Data":"e05be417e32006363014fd1aaf83746b4afd196b784e6ae64fefb0d2ec601873"} Jul 20 16:27:59.916100 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:59.916019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-xh8t9" event={"ID":"d711f142-170e-4fff-80c7-6c000dae7fa7","Type":"ContainerStarted","Data":"a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb"} Jul 20 16:27:59.916483 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:59.916146 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:27:59.934708 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:27:59.934668 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b54448d8-xh8t9" podStartSLOduration=1.263715485 podStartE2EDuration="1.934653747s" totalImagesPullingTime="670.938262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:27:59.932370261 +0000 UTC m=+828.137516002" watchObservedRunningTime="2026-07-20 16:27:59.934653747 +0000 UTC m=+828.139799488" Jul 20 16:28:10.922477 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:10.922445 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:28:15.533964 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.533925 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b54448d8-xh8t9"] Jul 20 16:28:15.534387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.534150 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b54448d8-xh8t9" podUID="d711f142-170e-4fff-80c7-6c000dae7fa7" containerName="manager" containerID="cri-o://a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb" gracePeriod=30 Jul 20 16:28:15.775196 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.775172 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:28:15.829541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.829462 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d711f142-170e-4fff-80c7-6c000dae7fa7-webhook-cert\") pod \"d711f142-170e-4fff-80c7-6c000dae7fa7\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " Jul 20 16:28:15.829541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.829525 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j289x\" (UniqueName: \"kubernetes.io/projected/d711f142-170e-4fff-80c7-6c000dae7fa7-kube-api-access-j289x\") pod \"d711f142-170e-4fff-80c7-6c000dae7fa7\" (UID: \"d711f142-170e-4fff-80c7-6c000dae7fa7\") " Jul 20 16:28:15.831653 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.831607 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d711f142-170e-4fff-80c7-6c000dae7fa7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d711f142-170e-4fff-80c7-6c000dae7fa7" (UID: "d711f142-170e-4fff-80c7-6c000dae7fa7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:28:15.831770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.831709 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d711f142-170e-4fff-80c7-6c000dae7fa7-kube-api-access-j289x" (OuterVolumeSpecName: "kube-api-access-j289x") pod "d711f142-170e-4fff-80c7-6c000dae7fa7" (UID: "d711f142-170e-4fff-80c7-6c000dae7fa7"). InnerVolumeSpecName "kube-api-access-j289x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:28:15.930384 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.930361 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d711f142-170e-4fff-80c7-6c000dae7fa7-webhook-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:28:15.930384 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.930383 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j289x\" (UniqueName: \"kubernetes.io/projected/d711f142-170e-4fff-80c7-6c000dae7fa7-kube-api-access-j289x\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:28:15.991425 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.991400 2582 generic.go:356] "Generic (PLEG): container finished" podID="d711f142-170e-4fff-80c7-6c000dae7fa7" containerID="a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb" exitCode=0 Jul 20 16:28:15.991528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.991439 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-xh8t9" event={"ID":"d711f142-170e-4fff-80c7-6c000dae7fa7","Type":"ContainerDied","Data":"a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb"} Jul 20 16:28:15.991528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.991447 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-xh8t9" Jul 20 16:28:15.991528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.991481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-xh8t9" event={"ID":"d711f142-170e-4fff-80c7-6c000dae7fa7","Type":"ContainerDied","Data":"e05be417e32006363014fd1aaf83746b4afd196b784e6ae64fefb0d2ec601873"} Jul 20 16:28:15.991528 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.991502 2582 scope.go:117] "RemoveContainer" containerID="a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb" Jul 20 16:28:15.999870 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:15.999853 2582 scope.go:117] "RemoveContainer" containerID="a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb" Jul 20 16:28:16.000126 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:28:16.000107 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb\": container with ID starting with a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb not found: ID does not exist" containerID="a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb" Jul 20 16:28:16.000191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.000136 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb"} err="failed to get container status \"a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb\": rpc error: code = NotFound desc = could not find container \"a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb\": container with ID starting with a4d0a31e7af3adb2fecf5b9597e8f25afaafad5468031b40ba41e49f9dddabdb not found: ID does not exist" Jul 20 16:28:16.017601 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.017578 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b54448d8-xh8t9"] Jul 20 16:28:16.021988 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.021969 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b54448d8-xh8t9"] Jul 20 16:28:16.329186 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.329144 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d711f142-170e-4fff-80c7-6c000dae7fa7" path="/var/lib/kubelet/pods/d711f142-170e-4fff-80c7-6c000dae7fa7/volumes" Jul 20 16:28:16.786260 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.786230 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b54448d8-q2b7v"] Jul 20 16:28:16.786604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.786573 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d711f142-170e-4fff-80c7-6c000dae7fa7" containerName="manager" Jul 20 16:28:16.786604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.786584 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d711f142-170e-4fff-80c7-6c000dae7fa7" containerName="manager" Jul 20 16:28:16.786705 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.786661 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d711f142-170e-4fff-80c7-6c000dae7fa7" containerName="manager" Jul 20 16:28:16.789806 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.789792 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:16.792319 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.792296 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 16:28:16.793375 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.793356 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 16:28:16.793455 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.793386 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8674p\"" Jul 20 16:28:16.797423 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.797399 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b54448d8-q2b7v"] Jul 20 16:28:16.835832 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.835807 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95d50ae2-17f6-41ac-a00b-cf1b6fa17260-webhook-cert\") pod \"maas-controller-67b54448d8-q2b7v\" (UID: \"95d50ae2-17f6-41ac-a00b-cf1b6fa17260\") " pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:16.835940 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.835846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lfg\" (UniqueName: \"kubernetes.io/projected/95d50ae2-17f6-41ac-a00b-cf1b6fa17260-kube-api-access-84lfg\") pod \"maas-controller-67b54448d8-q2b7v\" (UID: \"95d50ae2-17f6-41ac-a00b-cf1b6fa17260\") " pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:16.936533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.936510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95d50ae2-17f6-41ac-a00b-cf1b6fa17260-webhook-cert\") pod \"maas-controller-67b54448d8-q2b7v\" (UID: \"95d50ae2-17f6-41ac-a00b-cf1b6fa17260\") " pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:16.936651 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.936548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lfg\" (UniqueName: \"kubernetes.io/projected/95d50ae2-17f6-41ac-a00b-cf1b6fa17260-kube-api-access-84lfg\") pod \"maas-controller-67b54448d8-q2b7v\" (UID: \"95d50ae2-17f6-41ac-a00b-cf1b6fa17260\") " pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:16.938926 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.938910 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95d50ae2-17f6-41ac-a00b-cf1b6fa17260-webhook-cert\") pod \"maas-controller-67b54448d8-q2b7v\" (UID: \"95d50ae2-17f6-41ac-a00b-cf1b6fa17260\") " pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:16.944686 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:16.944666 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lfg\" (UniqueName: \"kubernetes.io/projected/95d50ae2-17f6-41ac-a00b-cf1b6fa17260-kube-api-access-84lfg\") pod \"maas-controller-67b54448d8-q2b7v\" (UID: \"95d50ae2-17f6-41ac-a00b-cf1b6fa17260\") " pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:17.100461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:17.100386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:17.230271 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:17.230250 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b54448d8-q2b7v"] Jul 20 16:28:17.232498 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:28:17.232472 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d50ae2_17f6_41ac_a00b_cf1b6fa17260.slice/crio-ecc9e2ec3ec365a4e5661c218e9cb4837e74d73a029e97d34cc45e3cbd428a50 WatchSource:0}: Error finding container ecc9e2ec3ec365a4e5661c218e9cb4837e74d73a029e97d34cc45e3cbd428a50: Status 404 returned error can't find the container with id ecc9e2ec3ec365a4e5661c218e9cb4837e74d73a029e97d34cc45e3cbd428a50 Jul 20 16:28:18.003886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:18.003851 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-q2b7v" event={"ID":"95d50ae2-17f6-41ac-a00b-cf1b6fa17260","Type":"ContainerStarted","Data":"ecc9e2ec3ec365a4e5661c218e9cb4837e74d73a029e97d34cc45e3cbd428a50"} Jul 20 16:28:19.010249 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:19.010210 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b54448d8-q2b7v" event={"ID":"95d50ae2-17f6-41ac-a00b-cf1b6fa17260","Type":"ContainerStarted","Data":"80e31cb65aeb99eadbb2cd7216caa75f6d58c342d857844c136a35de19004ce7"} Jul 20 16:28:19.010726 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:19.010274 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:28:19.026654 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:19.026583 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b54448d8-q2b7v" podStartSLOduration=2.243649338 podStartE2EDuration="3.026566247s" totalImagesPullingTime="782.916909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:28:19.025479093 +0000 UTC m=+847.230624836" watchObservedRunningTime="2026-07-20 16:28:19.026566247 +0000 UTC m=+847.231711990" Jul 20 16:28:30.017245 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:28:30.017215 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b54448d8-q2b7v" Jul 20 16:33:00.656263 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:00.656224 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2"] Jul 20 16:33:00.656849 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:00.656511 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" podUID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" containerName="manager" containerID="cri-o://a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67" gracePeriod=10 Jul 20 16:33:03.384885 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:03.384842 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" podUID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.27:8081/readyz\": dial tcp 10.134.0.27:8081: connect: connection refused" Jul 20 16:33:03.906183 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:03.906161 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:33:04.024809 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.024745 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-extensions-socket-volume\") pod \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " Jul 20 16:33:04.024929 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.024824 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgjd7\" (UniqueName: \"kubernetes.io/projected/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-kube-api-access-kgjd7\") pod \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\" (UID: \"3700f4fd-cfcd-4c24-81a9-e00c18d02d63\") " Jul 20 16:33:04.025057 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.025032 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3700f4fd-cfcd-4c24-81a9-e00c18d02d63" (UID: "3700f4fd-cfcd-4c24-81a9-e00c18d02d63"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:33:04.026675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.026649 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-kube-api-access-kgjd7" (OuterVolumeSpecName: "kube-api-access-kgjd7") pod "3700f4fd-cfcd-4c24-81a9-e00c18d02d63" (UID: "3700f4fd-cfcd-4c24-81a9-e00c18d02d63"). InnerVolumeSpecName "kube-api-access-kgjd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:33:04.125382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.125357 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-extensions-socket-volume\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:33:04.125382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.125379 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgjd7\" (UniqueName: \"kubernetes.io/projected/3700f4fd-cfcd-4c24-81a9-e00c18d02d63-kube-api-access-kgjd7\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:33:04.342683 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.342655 2582 generic.go:356] "Generic (PLEG): container finished" podID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" containerID="a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67" exitCode=0 Jul 20 16:33:04.342796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.342712 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" Jul 20 16:33:04.342796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.342734 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" event={"ID":"3700f4fd-cfcd-4c24-81a9-e00c18d02d63","Type":"ContainerDied","Data":"a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67"} Jul 20 16:33:04.342796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.342791 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2" event={"ID":"3700f4fd-cfcd-4c24-81a9-e00c18d02d63","Type":"ContainerDied","Data":"0d48cb6141382a659cb93cd05990132e17a42a0bedd7a9816ba1590c104e83f3"} Jul 20 16:33:04.342912 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.342820 2582 scope.go:117] "RemoveContainer" containerID="a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67" Jul 20 16:33:04.352529 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.352513 2582 scope.go:117] "RemoveContainer" containerID="a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67" Jul 20 16:33:04.352793 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:33:04.352772 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67\": container with ID starting with a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67 not found: ID does not exist" containerID="a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67" Jul 20 16:33:04.352867 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.352799 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67"} err="failed to get container status \"a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67\": rpc error: code = NotFound desc = could not find container \"a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67\": container with ID starting with a3c5322cd73d671c703864e8ad358697f311a86645c4232813e0c60c81ffdb67 not found: ID does not exist" Jul 20 16:33:04.361371 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.361345 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2"] Jul 20 16:33:04.363020 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:04.363001 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gkf2"] Jul 20 16:33:06.328455 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:33:06.328421 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" path="/var/lib/kubelet/pods/3700f4fd-cfcd-4c24-81a9-e00c18d02d63/volumes" Jul 20 16:34:01.844791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.844757 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd"] Jul 20 16:34:01.845260 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.845127 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" containerName="manager" Jul 20 16:34:01.845260 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.845138 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" containerName="manager" Jul 20 16:34:01.845260 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.845206 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3700f4fd-cfcd-4c24-81a9-e00c18d02d63" containerName="manager" Jul 20 16:34:01.848150 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.848134 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:01.850967 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.850945 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n542q\"" Jul 20 16:34:01.858510 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.858486 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd"] Jul 20 16:34:01.997400 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.997369 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08900eb9-f367-4b1a-b9c8-ba9c79b7c952-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bbsfd\" (UID: \"08900eb9-f367-4b1a-b9c8-ba9c79b7c952\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:01.997553 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:01.997423 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt85f\" (UniqueName: \"kubernetes.io/projected/08900eb9-f367-4b1a-b9c8-ba9c79b7c952-kube-api-access-kt85f\") pod \"kuadrant-operator-controller-manager-86878c769f-bbsfd\" (UID: \"08900eb9-f367-4b1a-b9c8-ba9c79b7c952\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.098476 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.098398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt85f\" (UniqueName: \"kubernetes.io/projected/08900eb9-f367-4b1a-b9c8-ba9c79b7c952-kube-api-access-kt85f\") pod \"kuadrant-operator-controller-manager-86878c769f-bbsfd\" (UID: \"08900eb9-f367-4b1a-b9c8-ba9c79b7c952\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.098639 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.098488 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08900eb9-f367-4b1a-b9c8-ba9c79b7c952-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bbsfd\" (UID: \"08900eb9-f367-4b1a-b9c8-ba9c79b7c952\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.098839 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.098823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08900eb9-f367-4b1a-b9c8-ba9c79b7c952-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bbsfd\" (UID: \"08900eb9-f367-4b1a-b9c8-ba9c79b7c952\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.106997 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.106972 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt85f\" (UniqueName: \"kubernetes.io/projected/08900eb9-f367-4b1a-b9c8-ba9c79b7c952-kube-api-access-kt85f\") pod \"kuadrant-operator-controller-manager-86878c769f-bbsfd\" (UID: \"08900eb9-f367-4b1a-b9c8-ba9c79b7c952\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.159061 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.159031 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.298439 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.298417 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd"] Jul 20 16:34:02.300150 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:34:02.300124 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08900eb9_f367_4b1a_b9c8_ba9c79b7c952.slice/crio-f2647943a3a63ca0efb353a4f180ffb7959e77f90db46207555d69c09477857e WatchSource:0}: Error finding container f2647943a3a63ca0efb353a4f180ffb7959e77f90db46207555d69c09477857e: Status 404 returned error can't find the container with id f2647943a3a63ca0efb353a4f180ffb7959e77f90db46207555d69c09477857e Jul 20 16:34:02.599757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.599722 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" event={"ID":"08900eb9-f367-4b1a-b9c8-ba9c79b7c952","Type":"ContainerStarted","Data":"8273f8d4639f0997e7cbe13150cf7fe0989a890b82bc3b50022880e338c17c21"} Jul 20 16:34:02.599935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.599771 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" event={"ID":"08900eb9-f367-4b1a-b9c8-ba9c79b7c952","Type":"ContainerStarted","Data":"f2647943a3a63ca0efb353a4f180ffb7959e77f90db46207555d69c09477857e"} Jul 20 16:34:02.599935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.599849 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:34:02.618936 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:02.618854 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" podStartSLOduration=1.618832264 podStartE2EDuration="1.618832264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:34:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:34:02.616694189 +0000 UTC m=+1190.821839930" watchObservedRunningTime="2026-07-20 16:34:02.618832264 +0000 UTC m=+1190.823978005" Jul 20 16:34:13.607386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:34:13.607350 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbsfd" Jul 20 16:36:58.306735 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.306699 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x"] Jul 20 16:36:58.310291 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.310271 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.312849 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.312828 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-294bb9f1-kserve-self-signed-certs\"" Jul 20 16:36:58.329036 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.329007 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x"] Jul 20 16:36:58.451498 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.451460 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tls-certs\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.451720 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.451526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-model-cache\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.451720 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.451685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tmp-dir\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.451720 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.451711 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rppg\" (UniqueName: \"kubernetes.io/projected/ab52fe6b-0420-4ceb-b506-ae1fe3973342-kube-api-access-5rppg\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.451924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.451765 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-home\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.451924 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.451869 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-dshm\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.552941 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.552905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-dshm\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.552941 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.552944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tls-certs\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.552982 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-model-cache\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.553015 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tmp-dir\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.553033 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rppg\" (UniqueName: \"kubernetes.io/projected/ab52fe6b-0420-4ceb-b506-ae1fe3973342-kube-api-access-5rppg\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.553065 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-home\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.553436 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-model-cache\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.553452 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tmp-dir\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.553565 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.553480 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-home\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.555861 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.555835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-dshm\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.556043 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.556026 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tls-certs\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.560764 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.560713 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rppg\" (UniqueName: \"kubernetes.io/projected/ab52fe6b-0420-4ceb-b506-ae1fe3973342-kube-api-access-5rppg\") pod \"e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.620640 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.620590 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:36:58.755129 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:58.755104 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x"] Jul 20 16:36:58.757912 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:36:58.757888 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab52fe6b_0420_4ceb_b506_ae1fe3973342.slice/crio-f346337ad0878e3dd9ef707728fb37871a8681b67d00e3d957429d63d1bf3fc9 WatchSource:0}: Error finding container f346337ad0878e3dd9ef707728fb37871a8681b67d00e3d957429d63d1bf3fc9: Status 404 returned error can't find the container with id f346337ad0878e3dd9ef707728fb37871a8681b67d00e3d957429d63d1bf3fc9 Jul 20 16:36:59.405607 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:59.405565 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" event={"ID":"ab52fe6b-0420-4ceb-b506-ae1fe3973342","Type":"ContainerStarted","Data":"1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877"} Jul 20 16:36:59.406010 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:59.405639 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" event={"ID":"ab52fe6b-0420-4ceb-b506-ae1fe3973342","Type":"ContainerStarted","Data":"f346337ad0878e3dd9ef707728fb37871a8681b67d00e3d957429d63d1bf3fc9"} Jul 20 16:36:59.423435 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:36:59.423389 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" podStartSLOduration=1.423374168 podStartE2EDuration="1.423374168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:36:59.422902855 +0000 UTC m=+1367.628048598" watchObservedRunningTime="2026-07-20 16:36:59.423374168 +0000 UTC m=+1367.628519957" Jul 20 16:37:08.620981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:08.620873 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:37:08.620981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:08.620932 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:37:08.633454 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:08.633422 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:37:09.465322 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:09.465294 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:37:27.572980 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:27.572936 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x"] Jul 20 16:37:42.595186 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.595143 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" podUID="ab52fe6b-0420-4ceb-b506-ae1fe3973342" containerName="main" containerID="cri-o://1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877" gracePeriod=45 Jul 20 16:37:42.863646 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.863553 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:37:42.979440 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979403 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-dshm\") pod \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " Jul 20 16:37:42.979644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979482 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rppg\" (UniqueName: \"kubernetes.io/projected/ab52fe6b-0420-4ceb-b506-ae1fe3973342-kube-api-access-5rppg\") pod \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " Jul 20 16:37:42.979644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979524 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-model-cache\") pod \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " Jul 20 16:37:42.979644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979566 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-home\") pod \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " Jul 20 16:37:42.979828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979721 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tmp-dir\") pod \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " Jul 20 16:37:42.979828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979759 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab52fe6b-0420-4ceb-b506-ae1fe3973342" (UID: "ab52fe6b-0420-4ceb-b506-ae1fe3973342"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:37:42.979828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979767 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tls-certs\") pod \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\" (UID: \"ab52fe6b-0420-4ceb-b506-ae1fe3973342\") " Jul 20 16:37:42.979828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979796 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-home" (OuterVolumeSpecName: "home") pod "ab52fe6b-0420-4ceb-b506-ae1fe3973342" (UID: "ab52fe6b-0420-4ceb-b506-ae1fe3973342"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:37:42.979998 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.979984 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab52fe6b-0420-4ceb-b506-ae1fe3973342" (UID: "ab52fe6b-0420-4ceb-b506-ae1fe3973342"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:37:42.980171 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.980152 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-model-cache\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:37:42.980258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.980176 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-home\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:37:42.980258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.980189 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tmp-dir\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:37:42.982010 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.981977 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab52fe6b-0420-4ceb-b506-ae1fe3973342" (UID: "ab52fe6b-0420-4ceb-b506-ae1fe3973342"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:37:42.982010 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.981978 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-dshm" (OuterVolumeSpecName: "dshm") pod "ab52fe6b-0420-4ceb-b506-ae1fe3973342" (UID: "ab52fe6b-0420-4ceb-b506-ae1fe3973342"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:37:42.982151 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:42.982063 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab52fe6b-0420-4ceb-b506-ae1fe3973342-kube-api-access-5rppg" (OuterVolumeSpecName: "kube-api-access-5rppg") pod "ab52fe6b-0420-4ceb-b506-ae1fe3973342" (UID: "ab52fe6b-0420-4ceb-b506-ae1fe3973342"). InnerVolumeSpecName "kube-api-access-5rppg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:37:43.080770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.080723 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rppg\" (UniqueName: \"kubernetes.io/projected/ab52fe6b-0420-4ceb-b506-ae1fe3973342-kube-api-access-5rppg\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:37:43.080770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.080760 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab52fe6b-0420-4ceb-b506-ae1fe3973342-tls-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:37:43.080770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.080770 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab52fe6b-0420-4ceb-b506-ae1fe3973342-dshm\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:37:43.622490 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.622449 2582 generic.go:356] "Generic (PLEG): container finished" podID="ab52fe6b-0420-4ceb-b506-ae1fe3973342" containerID="1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877" exitCode=0 Jul 20 16:37:43.622994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.622541 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" Jul 20 16:37:43.622994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.622545 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" event={"ID":"ab52fe6b-0420-4ceb-b506-ae1fe3973342","Type":"ContainerDied","Data":"1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877"} Jul 20 16:37:43.622994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.622602 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x" event={"ID":"ab52fe6b-0420-4ceb-b506-ae1fe3973342","Type":"ContainerDied","Data":"f346337ad0878e3dd9ef707728fb37871a8681b67d00e3d957429d63d1bf3fc9"} Jul 20 16:37:43.622994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.622642 2582 scope.go:117] "RemoveContainer" containerID="1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877" Jul 20 16:37:43.636303 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.636284 2582 scope.go:117] "RemoveContainer" containerID="1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877" Jul 20 16:37:43.636662 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:37:43.636638 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877\": container with ID starting with 1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877 not found: ID does not exist" containerID="1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877" Jul 20 16:37:43.636716 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.636673 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877"} err="failed to get container status \"1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877\": rpc error: code = NotFound desc = could not find container \"1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877\": container with ID starting with 1ca54441dabb196bf955378dd407b399b4345fa85bd338409563fe7f3a72f877 not found: ID does not exist" Jul 20 16:37:43.648386 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.648358 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x"] Jul 20 16:37:43.651897 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:43.651874 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-294bb9f1-kserve-c67f49f59-9rz2x"] Jul 20 16:37:44.329245 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:37:44.329204 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab52fe6b-0420-4ceb-b506-ae1fe3973342" path="/var/lib/kubelet/pods/ab52fe6b-0420-4ceb-b506-ae1fe3973342/volumes" Jul 20 16:41:18.870404 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.870323 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6"] Jul 20 16:41:18.870880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.870740 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab52fe6b-0420-4ceb-b506-ae1fe3973342" containerName="main" Jul 20 16:41:18.870880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.870754 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab52fe6b-0420-4ceb-b506-ae1fe3973342" containerName="main" Jul 20 16:41:18.870880 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.870815 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab52fe6b-0420-4ceb-b506-ae1fe3973342" containerName="main" Jul 20 16:41:18.874028 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.874005 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.876739 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.876715 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d56dc9f8-openshift-default-dockercfg-t87pl\"" Jul 20 16:41:18.894822 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.894783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6"] Jul 20 16:41:18.914671 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.914611 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-podinfo\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.914937 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.914907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-credential-socket\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915052 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.914958 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-token\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915052 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.915021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6rnx\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-kube-api-access-l6rnx\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915140 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.915055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istiod-ca-cert\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915140 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.915127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-certs\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915215 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.915172 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-socket\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915257 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.915213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-envoy\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:18.915257 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:18.915238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-data\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016240 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-certs\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016431 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016264 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-socket\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016431 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-envoy\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016431 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016355 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-data\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016431 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016412 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-podinfo\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016672 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-credential-socket\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016672 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-token\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016672 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6rnx\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-kube-api-access-l6rnx\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016672 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istiod-ca-cert\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.016908 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-certs\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.017009 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.016986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-socket\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.017103 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.017053 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-credential-socket\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.017103 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.017072 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-data\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.017403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.017386 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istiod-ca-cert\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.018713 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.018695 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-envoy\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.019223 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.019200 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-podinfo\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.025483 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.025455 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-token\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.025682 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.025664 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6rnx\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-kube-api-access-l6rnx\") pod \"e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.184378 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.184338 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:19.322772 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.322746 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6"] Jul 20 16:41:19.325239 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:41:19.325211 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9db3c6f_6fd3_4c5c_9425_d921b8259dd2.slice/crio-3284c37ec4862439f470244edc0a079032377b962b6caf4e4f61fb2c7b49ac21 WatchSource:0}: Error finding container 3284c37ec4862439f470244edc0a079032377b962b6caf4e4f61fb2c7b49ac21: Status 404 returned error can't find the container with id 3284c37ec4862439f470244edc0a079032377b962b6caf4e4f61fb2c7b49ac21 Jul 20 16:41:19.327469 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.327438 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:41:19.327583 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.327506 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:41:19.327583 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.327538 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:41:19.645045 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.644953 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" event={"ID":"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2","Type":"ContainerStarted","Data":"69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67"} Jul 20 16:41:19.645045 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.645011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" event={"ID":"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2","Type":"ContainerStarted","Data":"3284c37ec4862439f470244edc0a079032377b962b6caf4e4f61fb2c7b49ac21"} Jul 20 16:41:19.684932 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:19.684875 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" podStartSLOduration=1.684857365 podStartE2EDuration="1.684857365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:41:19.682572309 +0000 UTC m=+1627.887718061" watchObservedRunningTime="2026-07-20 16:41:19.684857365 +0000 UTC m=+1627.890003106" Jul 20 16:41:20.185194 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:20.185157 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:20.190089 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:20.190062 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:20.652820 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:20.652730 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:20.653737 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:20.653715 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:32.711246 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:32.711204 2582 generic.go:356] "Generic (PLEG): container finished" podID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerID="69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67" exitCode=0 Jul 20 16:41:32.711701 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:32.711279 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" event={"ID":"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2","Type":"ContainerDied","Data":"69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67"} Jul 20 16:41:32.711765 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:32.711750 2582 scope.go:117] "RemoveContainer" containerID="69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67" Jul 20 16:41:32.712883 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:32.712857 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:41:32.712952 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:32.712926 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:41:32.712995 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:32.712960 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:41:33.719046 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:33.719004 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" event={"ID":"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2","Type":"ContainerStarted","Data":"445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616"} Jul 20 16:41:34.184931 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:34.184902 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:34.186046 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:34.186021 2582 patch_prober.go:28] interesting pod/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 20 16:41:34.186167 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:34.186070 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 20 16:41:35.185563 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:35.185526 2582 patch_prober.go:28] interesting pod/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 20 16:41:35.186000 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:35.185583 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 20 16:41:35.652899 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:35.652803 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:36.185804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:36.185762 2582 patch_prober.go:28] interesting pod/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 20 16:41:36.186194 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:36.185820 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 20 16:41:37.185538 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:37.185507 2582 patch_prober.go:28] interesting pod/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 20 16:41:37.185770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:37.185733 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 20 16:41:38.188806 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:38.188769 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:41:38.189841 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:41:38.189820 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:42:01.946981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:01.946936 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6"] Jul 20 16:42:01.947815 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:01.947778 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" containerID="cri-o://445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616" gracePeriod=30 Jul 20 16:42:07.202532 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.202504 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:42:07.302462 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302359 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-podinfo\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302462 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302416 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-token\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302462 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302458 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istiod-ca-cert\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302507 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-certs\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302532 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-data\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302581 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6rnx\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-kube-api-access-l6rnx\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302642 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-envoy\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302714 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-socket\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.302796 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302760 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-credential-socket\") pod \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\" (UID: \"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2\") " Jul 20 16:42:07.303100 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.302845 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:42:07.303100 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303064 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:42:07.303100 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303082 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:42:07.303255 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303174 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:42:07.303308 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303278 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.303308 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303297 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-credential-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.303410 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303310 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istiod-ca-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.303410 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303325 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-workload-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.303556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.303529 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:42:07.305065 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.305030 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:42:07.305065 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.305037 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:42:07.305252 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.305093 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-kube-api-access-l6rnx" (OuterVolumeSpecName: "kube-api-access-l6rnx") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "kube-api-access-l6rnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:42:07.305252 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.305129 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" (UID: "d9db3c6f-6fd3-4c5c-9425-d921b8259dd2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:42:07.404598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.404547 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-podinfo\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.404598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.404588 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-token\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.404598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.404597 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-data\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.404598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.404605 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6rnx\" (UniqueName: \"kubernetes.io/projected/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-kube-api-access-l6rnx\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.404901 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.404644 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2-istio-envoy\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:42:07.888434 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.888393 2582 generic.go:356] "Generic (PLEG): container finished" podID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerID="445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616" exitCode=0 Jul 20 16:42:07.888676 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.888453 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" Jul 20 16:42:07.888676 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.888477 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" event={"ID":"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2","Type":"ContainerDied","Data":"445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616"} Jul 20 16:42:07.888676 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.888542 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6" event={"ID":"d9db3c6f-6fd3-4c5c-9425-d921b8259dd2","Type":"ContainerDied","Data":"3284c37ec4862439f470244edc0a079032377b962b6caf4e4f61fb2c7b49ac21"} Jul 20 16:42:07.888676 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.888566 2582 scope.go:117] "RemoveContainer" containerID="445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616" Jul 20 16:42:07.898505 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.898485 2582 scope.go:117] "RemoveContainer" containerID="69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67" Jul 20 16:42:07.907287 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.907268 2582 scope.go:117] "RemoveContainer" containerID="445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616" Jul 20 16:42:07.907574 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:42:07.907553 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616\": container with ID starting with 445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616 not found: ID does not exist" containerID="445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616" Jul 20 16:42:07.907670 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.907583 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616"} err="failed to get container status \"445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616\": rpc error: code = NotFound desc = could not find container \"445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616\": container with ID starting with 445080b3a67a781ed06e5e932c7ead2293cbe5e1695cec46eb4b7d080070d616 not found: ID does not exist" Jul 20 16:42:07.907670 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.907601 2582 scope.go:117] "RemoveContainer" containerID="69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67" Jul 20 16:42:07.907831 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:42:07.907817 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67\": container with ID starting with 69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67 not found: ID does not exist" containerID="69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67" Jul 20 16:42:07.907868 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.907836 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67"} err="failed to get container status \"69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67\": rpc error: code = NotFound desc = could not find container \"69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67\": container with ID starting with 69a8e71ef8efca8571bb61d58098866402e16f7248bf716dde77349b18fc0c67 not found: ID does not exist" Jul 20 16:42:07.911265 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.911244 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6"] Jul 20 16:42:07.915427 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:07.915403 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d56dc9f8-openshift-default-9746f665c-g6qf6"] Jul 20 16:42:08.329024 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:08.328990 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" path="/var/lib/kubelet/pods/d9db3c6f-6fd3-4c5c-9425-d921b8259dd2/volumes" Jul 20 16:42:46.483154 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483055 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t"] Jul 20 16:42:46.483751 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483601 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" Jul 20 16:42:46.483751 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483640 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" Jul 20 16:42:46.483751 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483660 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" Jul 20 16:42:46.483751 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483669 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" Jul 20 16:42:46.483971 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483788 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" Jul 20 16:42:46.483971 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.483802 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9db3c6f-6fd3-4c5c-9425-d921b8259dd2" containerName="istio-proxy" Jul 20 16:42:46.486982 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.486959 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.489474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.489451 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e7bb30ee-openshift-default-dockercfg-sdkvv\"" Jul 20 16:42:46.497326 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.497302 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t"] Jul 20 16:42:46.566271 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566229 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-credential-socket\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566441 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ff76f86-e0f4-4e03-96da-1961fbe60775-istiod-ca-cert\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566441 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566399 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-certs\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566522 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-data\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566522 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlzsp\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-kube-api-access-qlzsp\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-token\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566594 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-socket\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566719 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566652 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-envoy\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.566719 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.566684 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-podinfo\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.667524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzsp\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-kube-api-access-qlzsp\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.667524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667525 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-token\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.667829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-socket\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.667829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667597 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-envoy\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.667829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-podinfo\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.667829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-credential-socket\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668025 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ff76f86-e0f4-4e03-96da-1961fbe60775-istiod-ca-cert\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668025 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.667969 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-certs\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668133 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.668029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-data\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668133 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.668039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-socket\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668358 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.668332 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-certs\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668358 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.668352 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-data\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668493 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.668339 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-credential-socket\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.668582 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.668564 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ff76f86-e0f4-4e03-96da-1961fbe60775-istiod-ca-cert\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.670558 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.670535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-envoy\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.670897 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.670870 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-podinfo\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.675866 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.675844 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-token\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.676544 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.676522 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlzsp\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-kube-api-access-qlzsp\") pod \"e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.797683 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.797572 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:46.934247 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.934220 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t"] Jul 20 16:42:46.936872 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:42:46.936835 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff76f86_e0f4_4e03_96da_1961fbe60775.slice/crio-ea0fdc3419cf24353c02b916910a9c4009d28fe3ca24973918a9ae68d6988864 WatchSource:0}: Error finding container ea0fdc3419cf24353c02b916910a9c4009d28fe3ca24973918a9ae68d6988864: Status 404 returned error can't find the container with id ea0fdc3419cf24353c02b916910a9c4009d28fe3ca24973918a9ae68d6988864 Jul 20 16:42:46.939159 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.939122 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:42:46.939255 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.939202 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:42:46.939293 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:46.939269 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:42:47.078473 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:47.078382 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" event={"ID":"9ff76f86-e0f4-4e03-96da-1961fbe60775","Type":"ContainerStarted","Data":"9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5"} Jul 20 16:42:47.078473 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:47.078443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" event={"ID":"9ff76f86-e0f4-4e03-96da-1961fbe60775","Type":"ContainerStarted","Data":"ea0fdc3419cf24353c02b916910a9c4009d28fe3ca24973918a9ae68d6988864"} Jul 20 16:42:47.100314 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:47.100264 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podStartSLOduration=1.10024704 podStartE2EDuration="1.10024704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:42:47.097558278 +0000 UTC m=+1715.302704020" watchObservedRunningTime="2026-07-20 16:42:47.10024704 +0000 UTC m=+1715.305392782" Jul 20 16:42:47.798286 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:47.798249 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:47.803380 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:47.803355 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:48.084365 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:48.084280 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:42:48.085363 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:42:48.085346 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:03.889042 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:03.888216 2582 patch_prober.go:28] interesting pod/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:41774->10.134.0.41:15021: read: connection reset by peer" start-of-body= Jul 20 16:43:03.889042 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:03.888287 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:41774->10.134.0.41:15021: read: connection reset by peer" Jul 20 16:43:04.176429 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:04.176323 2582 generic.go:356] "Generic (PLEG): container finished" podID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerID="9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5" exitCode=0 Jul 20 16:43:04.176429 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:04.176377 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" event={"ID":"9ff76f86-e0f4-4e03-96da-1961fbe60775","Type":"ContainerDied","Data":"9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5"} Jul 20 16:43:04.176871 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:04.176825 2582 scope.go:117] "RemoveContainer" containerID="9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5" Jul 20 16:43:04.177915 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:04.177883 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:43:04.177987 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:04.177971 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:43:04.178031 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:04.178010 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:43:05.183465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:05.183426 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" event={"ID":"9ff76f86-e0f4-4e03-96da-1961fbe60775","Type":"ContainerStarted","Data":"3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d"} Jul 20 16:43:05.797975 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:05.797932 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:05.798237 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:05.798212 2582 patch_prober.go:28] interesting pod/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 20 16:43:05.798284 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:05.798265 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 20 16:43:06.798898 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:06.798862 2582 patch_prober.go:28] interesting pod/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 20 16:43:06.799281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:06.798938 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 20 16:43:07.798242 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:07.798194 2582 patch_prober.go:28] interesting pod/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 20 16:43:07.798443 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:07.798281 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 20 16:43:08.798747 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:08.798706 2582 patch_prober.go:28] interesting pod/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 20 16:43:08.799155 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:08.798768 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 20 16:43:09.801348 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:09.801313 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:09.801766 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:09.801579 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:09.802226 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:09.802207 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:30.477434 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:30.477387 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t"] Jul 20 16:43:30.477957 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:30.477791 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" containerID="cri-o://3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d" gracePeriod=30 Jul 20 16:43:35.751566 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.751494 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:35.851433 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851388 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ff76f86-e0f4-4e03-96da-1961fbe60775-istiod-ca-cert\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.851675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851456 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-data\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.851675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851474 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlzsp\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-kube-api-access-qlzsp\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.851675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851511 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-certs\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.851675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851569 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-credential-socket\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.851675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851595 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-envoy\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.851675 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851649 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-token\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.852017 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851690 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-socket\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.852017 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851715 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-podinfo\") pod \"9ff76f86-e0f4-4e03-96da-1961fbe60775\" (UID: \"9ff76f86-e0f4-4e03-96da-1961fbe60775\") " Jul 20 16:43:35.852017 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.851995 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ff76f86-e0f4-4e03-96da-1961fbe60775-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:43:35.852173 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.852079 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:43:35.852173 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.852109 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:43:35.852285 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.852265 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:43:35.852437 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.852402 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:43:35.853962 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.853938 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-kube-api-access-qlzsp" (OuterVolumeSpecName: "kube-api-access-qlzsp") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "kube-api-access-qlzsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:43:35.854185 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.854155 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:43:35.854297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.854228 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:43:35.854297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.854248 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ff76f86-e0f4-4e03-96da-1961fbe60775" (UID: "9ff76f86-e0f4-4e03-96da-1961fbe60775"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:43:35.953266 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953223 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-podinfo\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953266 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953258 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ff76f86-e0f4-4e03-96da-1961fbe60775-istiod-ca-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953266 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953267 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-data\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953278 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlzsp\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-kube-api-access-qlzsp\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953288 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953297 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-credential-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953305 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-envoy\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953313 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff76f86-e0f4-4e03-96da-1961fbe60775-istio-token\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:35.953556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:35.953320 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ff76f86-e0f4-4e03-96da-1961fbe60775-workload-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:43:36.345360 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.345326 2582 generic.go:356] "Generic (PLEG): container finished" podID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerID="3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d" exitCode=0 Jul 20 16:43:36.345541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.345400 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" Jul 20 16:43:36.345541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.345408 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" event={"ID":"9ff76f86-e0f4-4e03-96da-1961fbe60775","Type":"ContainerDied","Data":"3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d"} Jul 20 16:43:36.345541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.345503 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t" event={"ID":"9ff76f86-e0f4-4e03-96da-1961fbe60775","Type":"ContainerDied","Data":"ea0fdc3419cf24353c02b916910a9c4009d28fe3ca24973918a9ae68d6988864"} Jul 20 16:43:36.345541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.345533 2582 scope.go:117] "RemoveContainer" containerID="3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d" Jul 20 16:43:36.354667 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.354652 2582 scope.go:117] "RemoveContainer" containerID="9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5" Jul 20 16:43:36.363647 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.363608 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t"] Jul 20 16:43:36.364324 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.364305 2582 scope.go:117] "RemoveContainer" containerID="3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d" Jul 20 16:43:36.364730 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:43:36.364692 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d\": container with ID starting with 3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d not found: ID does not exist" containerID="3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d" Jul 20 16:43:36.364818 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.364738 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d"} err="failed to get container status \"3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d\": rpc error: code = NotFound desc = could not find container \"3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d\": container with ID starting with 3c05df106dcb3aaa102ad44db82619c03bcd10f5d42baeb397852549849f510d not found: ID does not exist" Jul 20 16:43:36.364818 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.364758 2582 scope.go:117] "RemoveContainer" containerID="9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5" Jul 20 16:43:36.365054 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:43:36.365030 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5\": container with ID starting with 9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5 not found: ID does not exist" containerID="9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5" Jul 20 16:43:36.365107 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.365064 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5"} err="failed to get container status \"9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5\": rpc error: code = NotFound desc = could not find container \"9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5\": container with ID starting with 9a890f03cec7b9c3bc07ffcb62f54a01645b237b8d7e2e2e6fb9103d2098e0d5 not found: ID does not exist" Jul 20 16:43:36.366425 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:36.366405 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e7bb30ee-openshift-default-55ccc6ccbf-8zg9t"] Jul 20 16:43:38.327839 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:43:38.327798 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" path="/var/lib/kubelet/pods/9ff76f86-e0f4-4e03-96da-1961fbe60775/volumes" Jul 20 16:45:55.504649 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.504596 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv"] Jul 20 16:45:55.505128 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.504998 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" Jul 20 16:45:55.505128 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.505013 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" Jul 20 16:45:55.505128 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.505111 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" Jul 20 16:45:55.508224 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.508207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.510923 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.510901 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-be2c65ba-openshift-default-dockercfg-5bxpq\"" Jul 20 16:45:55.520882 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.520858 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv"] Jul 20 16:45:55.550650 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414f95c7-665e-4886-b7bf-42c42a3362ac-istiod-ca-cert\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.550838 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550710 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-token\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.550838 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-credential-socket\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.550838 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550801 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-data\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.550838 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-podinfo\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.551027 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgn25\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-kube-api-access-qgn25\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.551027 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-socket\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.551027 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550954 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-envoy\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.551027 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.550977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-certs\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652204 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-envoy\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-certs\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652283 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414f95c7-665e-4886-b7bf-42c42a3362ac-istiod-ca-cert\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652332 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-token\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-credential-socket\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-data\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652734 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652423 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-podinfo\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652734 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgn25\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-kube-api-access-qgn25\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652734 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-socket\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652884 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652716 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-certs\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652884 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652791 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-credential-socket\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.652884 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652815 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-socket\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.653040 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.652950 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-data\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.653228 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.653206 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414f95c7-665e-4886-b7bf-42c42a3362ac-istiod-ca-cert\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.654556 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.654535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-envoy\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.654947 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.654928 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-podinfo\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.668932 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.668900 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-token\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.669223 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.669199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgn25\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-kube-api-access-qgn25\") pod \"e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.819137 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.819037 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:55.974343 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.974318 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv"] Jul 20 16:45:55.976960 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:45:55.976928 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414f95c7_665e_4886_b7bf_42c42a3362ac.slice/crio-fbf1c09c4accd074bf3cd28c88c9a108df5c740216dbf992144cf0a57900f50c WatchSource:0}: Error finding container fbf1c09c4accd074bf3cd28c88c9a108df5c740216dbf992144cf0a57900f50c: Status 404 returned error can't find the container with id fbf1c09c4accd074bf3cd28c88c9a108df5c740216dbf992144cf0a57900f50c Jul 20 16:45:55.980986 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.979899 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:45:55.980986 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.980056 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:45:55.980986 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:55.980274 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:45:56.015030 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:56.014984 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" event={"ID":"414f95c7-665e-4886-b7bf-42c42a3362ac","Type":"ContainerStarted","Data":"fbf1c09c4accd074bf3cd28c88c9a108df5c740216dbf992144cf0a57900f50c"} Jul 20 16:45:57.021374 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:57.021335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" event={"ID":"414f95c7-665e-4886-b7bf-42c42a3362ac","Type":"ContainerStarted","Data":"61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e"} Jul 20 16:45:57.045784 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:57.045722 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podStartSLOduration=2.04570455 podStartE2EDuration="2.04570455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:57.043846083 +0000 UTC m=+1905.248991844" watchObservedRunningTime="2026-07-20 16:45:57.04570455 +0000 UTC m=+1905.250850292" Jul 20 16:45:57.819990 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:57.819949 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:57.825252 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:57.825221 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:58.026945 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:58.026902 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:45:58.028104 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:45:58.028080 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:13.028588 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.028540 2582 patch_prober.go:28] interesting pod/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 20 16:46:13.029046 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.028614 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 20 16:46:13.104851 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.104814 2582 generic.go:356] "Generic (PLEG): container finished" podID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerID="61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e" exitCode=0 Jul 20 16:46:13.105020 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.104892 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" event={"ID":"414f95c7-665e-4886-b7bf-42c42a3362ac","Type":"ContainerDied","Data":"61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e"} Jul 20 16:46:13.105359 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.105345 2582 scope.go:117] "RemoveContainer" containerID="61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e" Jul 20 16:46:13.106382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.106357 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:46:13.106438 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.106426 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:46:13.106491 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:13.106478 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:46:14.111974 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:14.111936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" event={"ID":"414f95c7-665e-4886-b7bf-42c42a3362ac","Type":"ContainerStarted","Data":"3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d"} Jul 20 16:46:14.820147 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:14.820102 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:14.820363 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:14.820340 2582 patch_prober.go:28] interesting pod/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 20 16:46:14.820410 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:14.820389 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 20 16:46:15.819912 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:15.819867 2582 patch_prober.go:28] interesting pod/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 20 16:46:15.820317 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:15.819943 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 20 16:46:16.819558 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:16.819516 2582 patch_prober.go:28] interesting pod/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 20 16:46:16.819783 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:16.819588 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 20 16:46:17.819708 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:17.819662 2582 patch_prober.go:28] interesting pod/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 20 16:46:17.820104 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:17.819727 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 20 16:46:18.823684 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:18.823649 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:18.824143 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:18.823925 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:18.824729 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:18.824714 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:31.940276 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:31.940235 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv"] Jul 20 16:46:31.941123 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:31.941064 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" containerID="cri-o://3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d" gracePeriod=30 Jul 20 16:46:33.825947 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:33.825898 2582 patch_prober.go:28] interesting pod/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 16:46:33.826335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:33.825971 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 16:46:37.206405 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.206341 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:37.226981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.226935 2582 generic.go:356] "Generic (PLEG): container finished" podID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerID="3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d" exitCode=0 Jul 20 16:46:37.226981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.227006 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" Jul 20 16:46:37.226981 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.227022 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" event={"ID":"414f95c7-665e-4886-b7bf-42c42a3362ac","Type":"ContainerDied","Data":"3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d"} Jul 20 16:46:37.227393 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.227108 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv" event={"ID":"414f95c7-665e-4886-b7bf-42c42a3362ac","Type":"ContainerDied","Data":"fbf1c09c4accd074bf3cd28c88c9a108df5c740216dbf992144cf0a57900f50c"} Jul 20 16:46:37.227393 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.227139 2582 scope.go:117] "RemoveContainer" containerID="3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d" Jul 20 16:46:37.236891 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.236872 2582 scope.go:117] "RemoveContainer" containerID="61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e" Jul 20 16:46:37.245017 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.244996 2582 scope.go:117] "RemoveContainer" containerID="3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d" Jul 20 16:46:37.245295 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:46:37.245269 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d\": container with ID starting with 3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d not found: ID does not exist" containerID="3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d" Jul 20 16:46:37.245354 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.245309 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d"} err="failed to get container status \"3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d\": rpc error: code = NotFound desc = could not find container \"3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d\": container with ID starting with 3ea78deacdce4e87fe4aa97c511e3112b9de65eb0399f867cdcd0eaaa5092f5d not found: ID does not exist" Jul 20 16:46:37.245354 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.245338 2582 scope.go:117] "RemoveContainer" containerID="61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e" Jul 20 16:46:37.245576 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:46:37.245555 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e\": container with ID starting with 61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e not found: ID does not exist" containerID="61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e" Jul 20 16:46:37.245695 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.245579 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e"} err="failed to get container status \"61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e\": rpc error: code = NotFound desc = could not find container \"61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e\": container with ID starting with 61aaf84d70a4f45d044dac243c4a1912e0962a49cb7ba0c0c4d15a384573d22e not found: ID does not exist" Jul 20 16:46:37.259874 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.259852 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-socket\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.259994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.259887 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-podinfo\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.259994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.259930 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-data\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.259994 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.259950 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-envoy\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.260162 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260025 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-token\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.260162 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260062 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-credential-socket\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.260162 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260084 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgn25\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-kube-api-access-qgn25\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.260162 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260103 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:46:37.260368 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260187 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414f95c7-665e-4886-b7bf-42c42a3362ac-istiod-ca-cert\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.260368 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260220 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-certs\") pod \"414f95c7-665e-4886-b7bf-42c42a3362ac\" (UID: \"414f95c7-665e-4886-b7bf-42c42a3362ac\") " Jul 20 16:46:37.260601 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260576 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/414f95c7-665e-4886-b7bf-42c42a3362ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:46:37.260699 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260604 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.260699 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260673 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:46:37.260873 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260842 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:46:37.260873 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.260865 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:46:37.262210 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.262191 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:46:37.262210 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.262190 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:46:37.262718 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.262695 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-kube-api-access-qgn25" (OuterVolumeSpecName: "kube-api-access-qgn25") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "kube-api-access-qgn25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:46:37.262718 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.262703 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "414f95c7-665e-4886-b7bf-42c42a3362ac" (UID: "414f95c7-665e-4886-b7bf-42c42a3362ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:46:37.361900 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361864 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-token\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.361900 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361897 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-credential-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.361900 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361908 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgn25\" (UniqueName: \"kubernetes.io/projected/414f95c7-665e-4886-b7bf-42c42a3362ac-kube-api-access-qgn25\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.362148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361918 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414f95c7-665e-4886-b7bf-42c42a3362ac-istiod-ca-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.362148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361926 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-workload-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.362148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361938 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-podinfo\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.362148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361945 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-data\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.362148 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.361954 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414f95c7-665e-4886-b7bf-42c42a3362ac-istio-envoy\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:46:37.549697 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.549663 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv"] Jul 20 16:46:37.553563 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:37.553534 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-be2c65ba-openshift-default-85fb68bc5f-g6sgv"] Jul 20 16:46:38.328422 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:46:38.328375 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" path="/var/lib/kubelet/pods/414f95c7-665e-4886-b7bf-42c42a3362ac/volumes" Jul 20 16:47:06.905566 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905487 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt"] Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905880 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905891 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905914 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905921 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905929 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905936 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905992 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff76f86-e0f4-4e03-96da-1961fbe60775" containerName="istio-proxy" Jul 20 16:47:06.906037 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.905999 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" Jul 20 16:47:06.909315 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.909295 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:06.912185 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.912153 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3b97b792\"/\"openshift-service-ca.crt\"" Jul 20 16:47:06.912300 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.912190 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3b97b792\"/\"kube-root-ca.crt\"" Jul 20 16:47:06.913655 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.913599 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3b97b792\"/\"default-dockercfg-5mjxm\"" Jul 20 16:47:06.913753 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.913676 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3b97b792\"/\"e2e-lifecycle-model-3b97b792-kserve-self-signed-certs\"" Jul 20 16:47:06.920690 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.920668 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt"] Jul 20 16:47:06.936763 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.936735 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-tmp-dir\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:06.936920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.936777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhr85\" (UniqueName: \"kubernetes.io/projected/bb1ff609-1247-4472-8e1e-a192ef9a030c-kube-api-access-xhr85\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:06.936991 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.936926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-model-cache\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:06.936991 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.936967 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-dshm\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:06.937105 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.937021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-home\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:06.937105 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:06.937063 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb1ff609-1247-4472-8e1e-a192ef9a030c-tls-certs\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038011 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.037976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-model-cache\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038018 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-dshm\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038053 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-home\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038092 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb1ff609-1247-4472-8e1e-a192ef9a030c-tls-certs\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038140 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-tmp-dir\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038180 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhr85\" (UniqueName: \"kubernetes.io/projected/bb1ff609-1247-4472-8e1e-a192ef9a030c-kube-api-access-xhr85\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038469 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038450 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-model-cache\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038516 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038497 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-home\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.038593 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.038576 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-tmp-dir\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.040541 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.040515 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-dshm\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.040706 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.040685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb1ff609-1247-4472-8e1e-a192ef9a030c-tls-certs\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.046557 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.046535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhr85\" (UniqueName: \"kubernetes.io/projected/bb1ff609-1247-4472-8e1e-a192ef9a030c-kube-api-access-xhr85\") pod \"e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.219262 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.219225 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:07.349297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.349268 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt"] Jul 20 16:47:07.352218 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:47:07.352184 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb1ff609_1247_4472_8e1e_a192ef9a030c.slice/crio-17bf5583ef4da91809ad5bee0a46f5bad123542d6cbf1425b5ad0884c5b43a23 WatchSource:0}: Error finding container 17bf5583ef4da91809ad5bee0a46f5bad123542d6cbf1425b5ad0884c5b43a23: Status 404 returned error can't find the container with id 17bf5583ef4da91809ad5bee0a46f5bad123542d6cbf1425b5ad0884c5b43a23 Jul 20 16:47:07.372320 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:07.372290 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" event={"ID":"bb1ff609-1247-4472-8e1e-a192ef9a030c","Type":"ContainerStarted","Data":"17bf5583ef4da91809ad5bee0a46f5bad123542d6cbf1425b5ad0884c5b43a23"} Jul 20 16:47:08.378231 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:08.378186 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" event={"ID":"bb1ff609-1247-4472-8e1e-a192ef9a030c","Type":"ContainerStarted","Data":"603624f642ee6eb1e69152eb7b5c4dfc36dbd83198041db7a622c7cc8de20dd9"} Jul 20 16:47:08.395930 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:08.395878 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" podStartSLOduration=2.395860552 podStartE2EDuration="2.395860552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:47:08.394875995 +0000 UTC m=+1976.600021738" watchObservedRunningTime="2026-07-20 16:47:08.395860552 +0000 UTC m=+1976.601006294" Jul 20 16:47:17.220414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:17.220380 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:17.220812 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:17.220430 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:17.233635 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:17.233591 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:17.429882 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:17.429848 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:47:59.554086 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:47:59.554052 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt"] Jul 20 16:48:14.574052 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.574018 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" podUID="bb1ff609-1247-4472-8e1e-a192ef9a030c" containerName="main" containerID="cri-o://603624f642ee6eb1e69152eb7b5c4dfc36dbd83198041db7a622c7cc8de20dd9" gracePeriod=45 Jul 20 16:48:14.690511 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.690484 2582 generic.go:356] "Generic (PLEG): container finished" podID="bb1ff609-1247-4472-8e1e-a192ef9a030c" containerID="603624f642ee6eb1e69152eb7b5c4dfc36dbd83198041db7a622c7cc8de20dd9" exitCode=0 Jul 20 16:48:14.690638 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.690562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" event={"ID":"bb1ff609-1247-4472-8e1e-a192ef9a030c","Type":"ContainerDied","Data":"603624f642ee6eb1e69152eb7b5c4dfc36dbd83198041db7a622c7cc8de20dd9"} Jul 20 16:48:14.810639 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.810593 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:48:14.918088 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918016 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-model-cache\") pod \"bb1ff609-1247-4472-8e1e-a192ef9a030c\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " Jul 20 16:48:14.918088 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918060 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-home\") pod \"bb1ff609-1247-4472-8e1e-a192ef9a030c\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " Jul 20 16:48:14.918281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918108 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb1ff609-1247-4472-8e1e-a192ef9a030c-tls-certs\") pod \"bb1ff609-1247-4472-8e1e-a192ef9a030c\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " Jul 20 16:48:14.918281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918148 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-dshm\") pod \"bb1ff609-1247-4472-8e1e-a192ef9a030c\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " Jul 20 16:48:14.918281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918216 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-tmp-dir\") pod \"bb1ff609-1247-4472-8e1e-a192ef9a030c\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " Jul 20 16:48:14.918281 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918255 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhr85\" (UniqueName: \"kubernetes.io/projected/bb1ff609-1247-4472-8e1e-a192ef9a030c-kube-api-access-xhr85\") pod \"bb1ff609-1247-4472-8e1e-a192ef9a030c\" (UID: \"bb1ff609-1247-4472-8e1e-a192ef9a030c\") " Jul 20 16:48:14.918517 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918293 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb1ff609-1247-4472-8e1e-a192ef9a030c" (UID: "bb1ff609-1247-4472-8e1e-a192ef9a030c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:14.918517 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918322 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-home" (OuterVolumeSpecName: "home") pod "bb1ff609-1247-4472-8e1e-a192ef9a030c" (UID: "bb1ff609-1247-4472-8e1e-a192ef9a030c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:14.918517 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918455 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb1ff609-1247-4472-8e1e-a192ef9a030c" (UID: "bb1ff609-1247-4472-8e1e-a192ef9a030c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:14.918661 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918559 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-tmp-dir\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:48:14.918661 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918573 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-model-cache\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:48:14.918661 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.918582 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-home\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:48:14.920269 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.920241 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1ff609-1247-4472-8e1e-a192ef9a030c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb1ff609-1247-4472-8e1e-a192ef9a030c" (UID: "bb1ff609-1247-4472-8e1e-a192ef9a030c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:14.920375 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.920282 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-dshm" (OuterVolumeSpecName: "dshm") pod "bb1ff609-1247-4472-8e1e-a192ef9a030c" (UID: "bb1ff609-1247-4472-8e1e-a192ef9a030c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:14.920375 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:14.920309 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1ff609-1247-4472-8e1e-a192ef9a030c-kube-api-access-xhr85" (OuterVolumeSpecName: "kube-api-access-xhr85") pod "bb1ff609-1247-4472-8e1e-a192ef9a030c" (UID: "bb1ff609-1247-4472-8e1e-a192ef9a030c"). InnerVolumeSpecName "kube-api-access-xhr85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:48:15.019580 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.019559 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb1ff609-1247-4472-8e1e-a192ef9a030c-tls-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:48:15.019580 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.019576 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb1ff609-1247-4472-8e1e-a192ef9a030c-dshm\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:48:15.019580 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.019585 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhr85\" (UniqueName: \"kubernetes.io/projected/bb1ff609-1247-4472-8e1e-a192ef9a030c-kube-api-access-xhr85\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:48:15.696418 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.696391 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" Jul 20 16:48:15.696824 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.696389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt" event={"ID":"bb1ff609-1247-4472-8e1e-a192ef9a030c","Type":"ContainerDied","Data":"17bf5583ef4da91809ad5bee0a46f5bad123542d6cbf1425b5ad0884c5b43a23"} Jul 20 16:48:15.696824 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.696530 2582 scope.go:117] "RemoveContainer" containerID="603624f642ee6eb1e69152eb7b5c4dfc36dbd83198041db7a622c7cc8de20dd9" Jul 20 16:48:15.727297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.727276 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt"] Jul 20 16:48:15.730857 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:15.730836 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3b97b792/e2e-lifecycle-model-3b97b792-kserve-688b79dc67-9pxkt"] Jul 20 16:48:16.327488 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:48:16.327459 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1ff609-1247-4472-8e1e-a192ef9a030c" path="/var/lib/kubelet/pods/bb1ff609-1247-4472-8e1e-a192ef9a030c/volumes" Jul 20 16:49:39.572075 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.572045 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7"] Jul 20 16:49:39.572465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.572417 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1ff609-1247-4472-8e1e-a192ef9a030c" containerName="main" Jul 20 16:49:39.572465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.572428 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1ff609-1247-4472-8e1e-a192ef9a030c" containerName="main" Jul 20 16:49:39.572549 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.572496 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1ff609-1247-4472-8e1e-a192ef9a030c" containerName="main" Jul 20 16:49:39.572549 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.572505 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="414f95c7-665e-4886-b7bf-42c42a3362ac" containerName="istio-proxy" Jul 20 16:49:39.575458 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.575441 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.578099 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.578082 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2acd56-openshift-default-dockercfg-rc655\"" Jul 20 16:49:39.586091 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.586069 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7"] Jul 20 16:49:39.724854 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.724823 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-socket\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.724854 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.724856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48e9020c-5653-4530-8d9b-4dd4827d944b-istiod-ca-cert\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725047 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.724880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-data\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725047 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.724955 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-certs\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725047 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.725001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-podinfo\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725047 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.725027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-envoy\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725185 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.725046 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2j5g\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-kube-api-access-w2j5g\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725185 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.725112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-token\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.725185 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.725160 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-credential-socket\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826597 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-socket\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826688 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48e9020c-5653-4530-8d9b-4dd4827d944b-istiod-ca-cert\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-data\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-certs\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-podinfo\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826844 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-envoy\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826869 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2j5g\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-kube-api-access-w2j5g\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.826914 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826907 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-token\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.827210 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.826938 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-credential-socket\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.827210 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.827105 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-socket\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.827380 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.827356 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-credential-socket\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.827468 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.827457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-certs\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.827538 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.827512 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48e9020c-5653-4530-8d9b-4dd4827d944b-istiod-ca-cert\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.827656 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.827638 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-data\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.829669 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.829646 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-envoy\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.829877 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.829859 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-podinfo\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.835817 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.835793 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-token\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.836104 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.836084 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2j5g\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-kube-api-access-w2j5g\") pod \"e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:39.885056 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:39.885034 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:40.015988 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.015964 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7"] Jul 20 16:49:40.017928 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:49:40.017903 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e9020c_5653_4530_8d9b_4dd4827d944b.slice/crio-3cbae761ba8ff5fc127e7cb80c62324518c2c7b9593304231b52bb9fe6749182 WatchSource:0}: Error finding container 3cbae761ba8ff5fc127e7cb80c62324518c2c7b9593304231b52bb9fe6749182: Status 404 returned error can't find the container with id 3cbae761ba8ff5fc127e7cb80c62324518c2c7b9593304231b52bb9fe6749182 Jul 20 16:49:40.020124 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.020097 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:49:40.020179 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.020160 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:49:40.020213 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.020190 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:49:40.081129 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.081065 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" event={"ID":"48e9020c-5653-4530-8d9b-4dd4827d944b","Type":"ContainerStarted","Data":"5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c"} Jul 20 16:49:40.081129 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.081114 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" event={"ID":"48e9020c-5653-4530-8d9b-4dd4827d944b","Type":"ContainerStarted","Data":"3cbae761ba8ff5fc127e7cb80c62324518c2c7b9593304231b52bb9fe6749182"} Jul 20 16:49:40.103068 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.103025 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" podStartSLOduration=1.103011003 podStartE2EDuration="1.103011003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:49:40.100885463 +0000 UTC m=+2128.306031203" watchObservedRunningTime="2026-07-20 16:49:40.103011003 +0000 UTC m=+2128.308156743" Jul 20 16:49:40.885679 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:40.885639 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:41.889482 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:41.889452 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:42.090144 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:42.090111 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:42.090971 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:42.090931 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:55.156191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:55.156092 2582 generic.go:356] "Generic (PLEG): container finished" podID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerID="5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c" exitCode=0 Jul 20 16:49:55.156191 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:55.156133 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" event={"ID":"48e9020c-5653-4530-8d9b-4dd4827d944b","Type":"ContainerDied","Data":"5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c"} Jul 20 16:49:55.156581 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:55.156477 2582 scope.go:117] "RemoveContainer" containerID="5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c" Jul 20 16:49:55.157530 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:55.157499 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:49:55.157660 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:55.157638 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:49:55.157706 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:55.157693 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:49:56.163051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:56.163016 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" event={"ID":"48e9020c-5653-4530-8d9b-4dd4827d944b","Type":"ContainerStarted","Data":"8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc"} Jul 20 16:49:56.885394 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:56.885354 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:56.886509 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:56.886482 2582 patch_prober.go:28] interesting pod/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 20 16:49:56.886643 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:56.886542 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 20 16:49:57.090883 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:57.090848 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:57.886003 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:57.885967 2582 patch_prober.go:28] interesting pod/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 20 16:49:57.886406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:57.886023 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 20 16:49:58.885858 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:58.885818 2582 patch_prober.go:28] interesting pod/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 20 16:49:58.886048 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:58.885887 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 20 16:49:59.889807 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:59.889780 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:49:59.890605 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:49:59.890586 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:50:18.906067 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.905993 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4"] Jul 20 16:50:18.909966 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.909945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:18.912812 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.912790 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-59dc77\"/\"kube-root-ca.crt\"" Jul 20 16:50:18.913866 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.913836 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-59dc77\"/\"openshift-service-ca.crt\"" Jul 20 16:50:18.913978 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.913960 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-59dc77\"/\"test-model-59dc77-kserve-self-signed-certs\"" Jul 20 16:50:18.914047 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.914019 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-59dc77\"/\"default-dockercfg-vrn8d\"" Jul 20 16:50:18.917499 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.917477 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4"] Jul 20 16:50:18.996379 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.996351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-tmp-dir\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:18.996481 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.996381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-home\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:18.996481 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.996412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-model-cache\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:18.996481 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.996469 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66968\" (UniqueName: \"kubernetes.io/projected/441914f0-105f-4b5f-907b-3c5af0758466-kube-api-access-66968\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:18.996604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.996545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-dshm\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:18.996604 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:18.996572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441914f0-105f-4b5f-907b-3c5af0758466-tls-certs\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097292 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097262 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-tmp-dir\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097429 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-home\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097429 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-model-cache\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097540 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097517 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66968\" (UniqueName: \"kubernetes.io/projected/441914f0-105f-4b5f-907b-3c5af0758466-kube-api-access-66968\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097641 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-dshm\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097779 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097660 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441914f0-105f-4b5f-907b-3c5af0758466-tls-certs\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097779 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097689 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-tmp-dir\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097779 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097729 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-model-cache\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.097779 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.097758 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-home\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.099755 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.099728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-dshm\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.100014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.099999 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441914f0-105f-4b5f-907b-3c5af0758466-tls-certs\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.107024 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.107004 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66968\" (UniqueName: \"kubernetes.io/projected/441914f0-105f-4b5f-907b-3c5af0758466-kube-api-access-66968\") pod \"test-model-59dc77-kserve-6d889cf84c-s7sm4\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.219673 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.219652 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:19.352595 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:19.352571 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4"] Jul 20 16:50:19.354751 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:50:19.354728 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod441914f0_105f_4b5f_907b_3c5af0758466.slice/crio-c138d71f35af9fa56a438fb9f06eaf50a1df2460374ac8bd174a57c9680fedca WatchSource:0}: Error finding container c138d71f35af9fa56a438fb9f06eaf50a1df2460374ac8bd174a57c9680fedca: Status 404 returned error can't find the container with id c138d71f35af9fa56a438fb9f06eaf50a1df2460374ac8bd174a57c9680fedca Jul 20 16:50:20.277804 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:20.277768 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" event={"ID":"441914f0-105f-4b5f-907b-3c5af0758466","Type":"ContainerStarted","Data":"57f409dc17e99fca1883cf82a6bf41e71e71ccbaff7401d07964133cbbf65711"} Jul 20 16:50:20.278146 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:20.277812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" event={"ID":"441914f0-105f-4b5f-907b-3c5af0758466","Type":"ContainerStarted","Data":"c138d71f35af9fa56a438fb9f06eaf50a1df2460374ac8bd174a57c9680fedca"} Jul 20 16:50:20.302043 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:20.301998 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" podStartSLOduration=2.301982432 podStartE2EDuration="2.301982432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:20.301642821 +0000 UTC m=+2168.506788567" watchObservedRunningTime="2026-07-20 16:50:20.301982432 +0000 UTC m=+2168.507128172" Jul 20 16:50:29.220371 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:29.220341 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:29.220851 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:29.220409 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:29.236093 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:29.236071 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:29.328716 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:29.328691 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:50:45.226448 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:50:45.226415 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4"] Jul 20 16:51:00.245973 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.245909 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" podUID="441914f0-105f-4b5f-907b-3c5af0758466" containerName="main" containerID="cri-o://57f409dc17e99fca1883cf82a6bf41e71e71ccbaff7401d07964133cbbf65711" gracePeriod=45 Jul 20 16:51:00.461368 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.461333 2582 generic.go:356] "Generic (PLEG): container finished" podID="441914f0-105f-4b5f-907b-3c5af0758466" containerID="57f409dc17e99fca1883cf82a6bf41e71e71ccbaff7401d07964133cbbf65711" exitCode=0 Jul 20 16:51:00.461524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.461402 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" event={"ID":"441914f0-105f-4b5f-907b-3c5af0758466","Type":"ContainerDied","Data":"57f409dc17e99fca1883cf82a6bf41e71e71ccbaff7401d07964133cbbf65711"} Jul 20 16:51:00.488280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.488257 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:51:00.545217 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545146 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66968\" (UniqueName: \"kubernetes.io/projected/441914f0-105f-4b5f-907b-3c5af0758466-kube-api-access-66968\") pod \"441914f0-105f-4b5f-907b-3c5af0758466\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " Jul 20 16:51:00.545217 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545187 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-home\") pod \"441914f0-105f-4b5f-907b-3c5af0758466\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " Jul 20 16:51:00.545217 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545206 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-tmp-dir\") pod \"441914f0-105f-4b5f-907b-3c5af0758466\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " Jul 20 16:51:00.545417 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545231 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-model-cache\") pod \"441914f0-105f-4b5f-907b-3c5af0758466\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " Jul 20 16:51:00.545417 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545316 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-dshm\") pod \"441914f0-105f-4b5f-907b-3c5af0758466\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " Jul 20 16:51:00.545417 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545349 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441914f0-105f-4b5f-907b-3c5af0758466-tls-certs\") pod \"441914f0-105f-4b5f-907b-3c5af0758466\" (UID: \"441914f0-105f-4b5f-907b-3c5af0758466\") " Jul 20 16:51:00.545533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545431 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-home" (OuterVolumeSpecName: "home") pod "441914f0-105f-4b5f-907b-3c5af0758466" (UID: "441914f0-105f-4b5f-907b-3c5af0758466"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:00.545533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545448 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "441914f0-105f-4b5f-907b-3c5af0758466" (UID: "441914f0-105f-4b5f-907b-3c5af0758466"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:00.545533 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545518 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-model-cache" (OuterVolumeSpecName: "model-cache") pod "441914f0-105f-4b5f-907b-3c5af0758466" (UID: "441914f0-105f-4b5f-907b-3c5af0758466"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:00.545703 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545687 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-home\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:00.545752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545706 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-tmp-dir\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:00.545752 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.545717 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-model-cache\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:00.547459 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.547434 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441914f0-105f-4b5f-907b-3c5af0758466-kube-api-access-66968" (OuterVolumeSpecName: "kube-api-access-66968") pod "441914f0-105f-4b5f-907b-3c5af0758466" (UID: "441914f0-105f-4b5f-907b-3c5af0758466"). InnerVolumeSpecName "kube-api-access-66968". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:51:00.547459 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.547449 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/441914f0-105f-4b5f-907b-3c5af0758466-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "441914f0-105f-4b5f-907b-3c5af0758466" (UID: "441914f0-105f-4b5f-907b-3c5af0758466"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:51:00.547637 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.547520 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-dshm" (OuterVolumeSpecName: "dshm") pod "441914f0-105f-4b5f-907b-3c5af0758466" (UID: "441914f0-105f-4b5f-907b-3c5af0758466"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:00.647180 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.647148 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66968\" (UniqueName: \"kubernetes.io/projected/441914f0-105f-4b5f-907b-3c5af0758466-kube-api-access-66968\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:00.647180 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.647176 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441914f0-105f-4b5f-907b-3c5af0758466-dshm\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:00.647335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:00.647187 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441914f0-105f-4b5f-907b-3c5af0758466-tls-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:01.466728 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:01.466696 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" Jul 20 16:51:01.466728 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:01.466719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4" event={"ID":"441914f0-105f-4b5f-907b-3c5af0758466","Type":"ContainerDied","Data":"c138d71f35af9fa56a438fb9f06eaf50a1df2460374ac8bd174a57c9680fedca"} Jul 20 16:51:01.467199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:01.466769 2582 scope.go:117] "RemoveContainer" containerID="57f409dc17e99fca1883cf82a6bf41e71e71ccbaff7401d07964133cbbf65711" Jul 20 16:51:01.494258 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:01.494236 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4"] Jul 20 16:51:01.502455 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:01.502433 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-59dc77/test-model-59dc77-kserve-6d889cf84c-s7sm4"] Jul 20 16:51:02.328227 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:02.328185 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="441914f0-105f-4b5f-907b-3c5af0758466" path="/var/lib/kubelet/pods/441914f0-105f-4b5f-907b-3c5af0758466/volumes" Jul 20 16:51:06.885520 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:06.885482 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7"] Jul 20 16:51:06.885966 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:06.885696 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" containerID="cri-o://8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc" gracePeriod=30 Jul 20 16:51:12.139404 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.139383 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:51:12.245858 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.245824 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-token\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.245871 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-podinfo\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.245907 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-data\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.245943 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2j5g\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-kube-api-access-w2j5g\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.245974 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-socket\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246016 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-certs\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246051 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246044 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-credential-socket\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246351 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246093 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48e9020c-5653-4530-8d9b-4dd4827d944b-istiod-ca-cert\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246351 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246148 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-envoy\") pod \"48e9020c-5653-4530-8d9b-4dd4827d944b\" (UID: \"48e9020c-5653-4530-8d9b-4dd4827d944b\") " Jul 20 16:51:12.246351 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246307 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:12.246351 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246328 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:12.246551 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246453 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:12.246607 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246562 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.246607 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246579 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-workload-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.246607 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246593 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-credential-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.246781 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246634 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e9020c-5653-4530-8d9b-4dd4827d944b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:51:12.246859 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.246837 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-data" (OuterVolumeSpecName: "istio-data") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:12.248297 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.248271 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:12.248464 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.248444 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-token" (OuterVolumeSpecName: "istio-token") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:51:12.248536 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.248482 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:51:12.248597 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.248575 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-kube-api-access-w2j5g" (OuterVolumeSpecName: "kube-api-access-w2j5g") pod "48e9020c-5653-4530-8d9b-4dd4827d944b" (UID: "48e9020c-5653-4530-8d9b-4dd4827d944b"). InnerVolumeSpecName "kube-api-access-w2j5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:51:12.347647 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.347611 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48e9020c-5653-4530-8d9b-4dd4827d944b-istiod-ca-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.347647 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.347648 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-envoy\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.347774 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.347657 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-token\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.347774 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.347666 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-podinfo\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.347774 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.347673 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48e9020c-5653-4530-8d9b-4dd4827d944b-istio-data\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.347774 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.347682 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2j5g\" (UniqueName: \"kubernetes.io/projected/48e9020c-5653-4530-8d9b-4dd4827d944b-kube-api-access-w2j5g\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:51:12.517954 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.517890 2582 generic.go:356] "Generic (PLEG): container finished" podID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerID="8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc" exitCode=0 Jul 20 16:51:12.517954 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.517940 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" event={"ID":"48e9020c-5653-4530-8d9b-4dd4827d944b","Type":"ContainerDied","Data":"8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc"} Jul 20 16:51:12.518119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.517960 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" Jul 20 16:51:12.518119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.517981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7" event={"ID":"48e9020c-5653-4530-8d9b-4dd4827d944b","Type":"ContainerDied","Data":"3cbae761ba8ff5fc127e7cb80c62324518c2c7b9593304231b52bb9fe6749182"} Jul 20 16:51:12.518119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.518001 2582 scope.go:117] "RemoveContainer" containerID="8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc" Jul 20 16:51:12.525810 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.525787 2582 scope.go:117] "RemoveContainer" containerID="5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c" Jul 20 16:51:12.532612 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.532596 2582 scope.go:117] "RemoveContainer" containerID="8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc" Jul 20 16:51:12.532857 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:51:12.532840 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc\": container with ID starting with 8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc not found: ID does not exist" containerID="8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc" Jul 20 16:51:12.532903 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.532865 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc"} err="failed to get container status \"8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc\": rpc error: code = NotFound desc = could not find container \"8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc\": container with ID starting with 8dec6e5aafb1916ccf28c32190c704d90023e0b8867b871ccdb6acfdc9c5d3fc not found: ID does not exist" Jul 20 16:51:12.532903 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.532881 2582 scope.go:117] "RemoveContainer" containerID="5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c" Jul 20 16:51:12.533116 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:51:12.533095 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c\": container with ID starting with 5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c not found: ID does not exist" containerID="5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c" Jul 20 16:51:12.533156 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.533126 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c"} err="failed to get container status \"5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c\": rpc error: code = NotFound desc = could not find container \"5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c\": container with ID starting with 5072a9d561767bb1642cf6136426157f08870ab4668ac5c2a0a389ff3853181c not found: ID does not exist" Jul 20 16:51:12.535983 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.535961 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7"] Jul 20 16:51:12.564152 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:12.564129 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2acd56-openshift-default-657447c499-p7sg7"] Jul 20 16:51:14.327800 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:14.327768 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" path="/var/lib/kubelet/pods/48e9020c-5653-4530-8d9b-4dd4827d944b/volumes" Jul 20 16:51:29.650173 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650139 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj"] Jul 20 16:51:29.650690 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650671 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" Jul 20 16:51:29.650775 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650692 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" Jul 20 16:51:29.650775 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650702 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441914f0-105f-4b5f-907b-3c5af0758466" containerName="main" Jul 20 16:51:29.650775 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650709 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="441914f0-105f-4b5f-907b-3c5af0758466" containerName="main" Jul 20 16:51:29.650949 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650815 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" Jul 20 16:51:29.650949 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650829 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" Jul 20 16:51:29.650949 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.650838 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="441914f0-105f-4b5f-907b-3c5af0758466" containerName="main" Jul 20 16:51:29.653964 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.653943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.657181 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.657162 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7498b7-openshift-default-dockercfg-zwfrr\"" Jul 20 16:51:29.666491 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.666465 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj"] Jul 20 16:51:29.791440 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791407 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-envoy\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791591 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791444 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-token\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791591 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-podinfo\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791591 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-socket\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791781 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-credential-socket\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791781 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791640 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-data\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791781 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791674 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt85c\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-kube-api-access-wt85c\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791781 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-certs\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.791781 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.791776 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39336b8f-895d-4601-a78f-ec2bf8627da3-istiod-ca-cert\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.892660 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892600 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-socket\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.892791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-credential-socket\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.892791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892697 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-data\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.892791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892722 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt85c\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-kube-api-access-wt85c\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.892791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-certs\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.892791 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892762 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39336b8f-895d-4601-a78f-ec2bf8627da3-istiod-ca-cert\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893030 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-envoy\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893030 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-token\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893030 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.892873 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-podinfo\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893184 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.893058 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-socket\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893184 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.893133 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-certs\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893275 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.893192 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-credential-socket\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893358 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.893339 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-data\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.893481 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.893462 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39336b8f-895d-4601-a78f-ec2bf8627da3-istiod-ca-cert\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.895071 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.895039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-envoy\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.895276 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.895258 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-podinfo\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.901218 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.901146 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-token\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.901549 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.901530 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt85c\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-kube-api-access-wt85c\") pod \"e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:29.963639 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:29.963602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:30.096673 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.096645 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj"] Jul 20 16:51:30.098462 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:51:30.098435 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39336b8f_895d_4601_a78f_ec2bf8627da3.slice/crio-fbf20bb6a1ab1093e6af08fafcbc5225ccfb95b74f9b665089fc44eff4facc20 WatchSource:0}: Error finding container fbf20bb6a1ab1093e6af08fafcbc5225ccfb95b74f9b665089fc44eff4facc20: Status 404 returned error can't find the container with id fbf20bb6a1ab1093e6af08fafcbc5225ccfb95b74f9b665089fc44eff4facc20 Jul 20 16:51:30.100633 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.100582 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:51:30.100706 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.100682 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:51:30.100741 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.100709 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:51:30.607370 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.607334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" event={"ID":"39336b8f-895d-4601-a78f-ec2bf8627da3","Type":"ContainerStarted","Data":"ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16"} Jul 20 16:51:30.607561 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.607385 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" event={"ID":"39336b8f-895d-4601-a78f-ec2bf8627da3","Type":"ContainerStarted","Data":"fbf20bb6a1ab1093e6af08fafcbc5225ccfb95b74f9b665089fc44eff4facc20"} Jul 20 16:51:30.631602 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.631556 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podStartSLOduration=1.6315424840000001 podStartE2EDuration="1.631542484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:51:30.630574295 +0000 UTC m=+2238.835720037" watchObservedRunningTime="2026-07-20 16:51:30.631542484 +0000 UTC m=+2238.836688225" Jul 20 16:51:30.964056 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:30.964022 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:31.968842 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:31.968811 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:32.616850 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:32.616817 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:32.617735 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:32.617715 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:47.618555 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.618520 2582 patch_prober.go:28] interesting pod/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 20 16:51:47.618967 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.618571 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 20 16:51:47.689145 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.689118 2582 generic.go:356] "Generic (PLEG): container finished" podID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerID="ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16" exitCode=0 Jul 20 16:51:47.689274 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.689169 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" event={"ID":"39336b8f-895d-4601-a78f-ec2bf8627da3","Type":"ContainerDied","Data":"ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16"} Jul 20 16:51:47.689488 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.689474 2582 scope.go:117] "RemoveContainer" containerID="ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16" Jul 20 16:51:47.690423 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.690396 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:51:47.690488 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.690466 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:51:47.690525 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:47.690500 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:51:48.695329 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:48.695292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" event={"ID":"39336b8f-895d-4601-a78f-ec2bf8627da3","Type":"ContainerStarted","Data":"4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a"} Jul 20 16:51:48.964737 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:48.964641 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:48.964920 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:48.964886 2582 patch_prober.go:28] interesting pod/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 20 16:51:48.964993 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:48.964958 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 20 16:51:49.964814 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:49.964783 2582 patch_prober.go:28] interesting pod/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 20 16:51:49.965213 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:49.964833 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 20 16:51:50.964317 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:50.964285 2582 patch_prober.go:28] interesting pod/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 20 16:51:50.964500 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:50.964340 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 20 16:51:51.964888 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:51.964852 2582 patch_prober.go:28] interesting pod/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 20 16:51:51.965288 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:51.964936 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 20 16:51:52.968373 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:52.968343 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:52.968797 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:52.968641 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:51:52.969407 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:51:52.969388 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:52:20.017437 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:20.017406 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj"] Jul 20 16:52:20.017976 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:20.017713 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" containerID="cri-o://4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a" gracePeriod=30 Jul 20 16:52:22.971316 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:22.971279 2582 patch_prober.go:28] interesting pod/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 16:52:22.971754 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:22.971350 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 16:52:25.261024 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.261002 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:52:25.275636 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275598 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-certs\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.275738 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275665 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-credential-socket\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.275738 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275700 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-data\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.275738 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275731 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt85c\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-kube-api-access-wt85c\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.275864 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275812 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-podinfo\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.275864 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275809 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:25.275864 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275842 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-token\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.275990 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275960 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39336b8f-895d-4601-a78f-ec2bf8627da3-istiod-ca-cert\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.276050 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276027 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-socket\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.276050 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.275961 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:25.276172 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276076 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-envoy\") pod \"39336b8f-895d-4601-a78f-ec2bf8627da3\" (UID: \"39336b8f-895d-4601-a78f-ec2bf8627da3\") " Jul 20 16:52:25.276300 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276275 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39336b8f-895d-4601-a78f-ec2bf8627da3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:52:25.276385 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276282 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:25.276467 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276440 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-data" (OuterVolumeSpecName: "istio-data") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:25.276577 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276464 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39336b8f-895d-4601-a78f-ec2bf8627da3-istiod-ca-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.276577 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276486 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.276577 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276497 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-workload-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.276577 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.276506 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-credential-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.278021 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.277967 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-token" (OuterVolumeSpecName: "istio-token") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:52:25.278021 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.277970 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:52:25.278021 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.277964 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-kube-api-access-wt85c" (OuterVolumeSpecName: "kube-api-access-wt85c") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "kube-api-access-wt85c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:52:25.278170 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.278128 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39336b8f-895d-4601-a78f-ec2bf8627da3" (UID: "39336b8f-895d-4601-a78f-ec2bf8627da3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:25.377598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.377573 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-envoy\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.377598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.377594 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-data\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.377598 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.377605 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt85c\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-kube-api-access-wt85c\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.377815 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.377735 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-podinfo\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.377815 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.377754 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39336b8f-895d-4601-a78f-ec2bf8627da3-istio-token\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:52:25.868952 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.868918 2582 generic.go:356] "Generic (PLEG): container finished" podID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerID="4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a" exitCode=0 Jul 20 16:52:25.869119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.868967 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" Jul 20 16:52:25.869119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.868996 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" event={"ID":"39336b8f-895d-4601-a78f-ec2bf8627da3","Type":"ContainerDied","Data":"4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a"} Jul 20 16:52:25.869119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.869050 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj" event={"ID":"39336b8f-895d-4601-a78f-ec2bf8627da3","Type":"ContainerDied","Data":"fbf20bb6a1ab1093e6af08fafcbc5225ccfb95b74f9b665089fc44eff4facc20"} Jul 20 16:52:25.869119 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.869073 2582 scope.go:117] "RemoveContainer" containerID="4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a" Jul 20 16:52:25.878998 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.878979 2582 scope.go:117] "RemoveContainer" containerID="ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16" Jul 20 16:52:25.886879 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.886862 2582 scope.go:117] "RemoveContainer" containerID="4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a" Jul 20 16:52:25.887139 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:52:25.887121 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a\": container with ID starting with 4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a not found: ID does not exist" containerID="4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a" Jul 20 16:52:25.887199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.887148 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a"} err="failed to get container status \"4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a\": rpc error: code = NotFound desc = could not find container \"4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a\": container with ID starting with 4b062d80d8a1283407bfd4d6ff6e6e0dab081df173a991d05d5d987beef5042a not found: ID does not exist" Jul 20 16:52:25.887199 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.887163 2582 scope.go:117] "RemoveContainer" containerID="ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16" Jul 20 16:52:25.887392 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:52:25.887374 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16\": container with ID starting with ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16 not found: ID does not exist" containerID="ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16" Jul 20 16:52:25.887449 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.887401 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16"} err="failed to get container status \"ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16\": rpc error: code = NotFound desc = could not find container \"ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16\": container with ID starting with ec6c6fbaf04115998743b6145d7440cf1b29435e775eb37e6431f40a1c0dcf16 not found: ID does not exist" Jul 20 16:52:25.893083 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.893061 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj"] Jul 20 16:52:25.897059 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:25.897040 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7498b7-openshift-default-65fcfc96d5-5jdvj"] Jul 20 16:52:26.327678 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:26.327640 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" path="/var/lib/kubelet/pods/39336b8f-895d-4601-a78f-ec2bf8627da3/volumes" Jul 20 16:52:47.537867 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.537837 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp"] Jul 20 16:52:47.545414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.539166 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" Jul 20 16:52:47.545414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.539201 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" Jul 20 16:52:47.545414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.539309 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" Jul 20 16:52:47.545414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.539321 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e9020c-5653-4530-8d9b-4dd4827d944b" containerName="istio-proxy" Jul 20 16:52:47.545414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.539877 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" Jul 20 16:52:47.545414 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.539905 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" Jul 20 16:52:47.548146 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.547374 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.550765 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.550745 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-42e123-openshift-default-dockercfg-dhbkc\"" Jul 20 16:52:47.554586 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.554563 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp"] Jul 20 16:52:47.664903 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.664874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-envoy\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665057 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.664913 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-certs\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665057 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.664995 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-credential-socket\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.665065 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-token\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.665096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcb7q\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-kube-api-access-bcb7q\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.665122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-podinfo\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.665143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-socket\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.665160 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-data\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.665239 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.665216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istiod-ca-cert\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.765839 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.765811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-credential-socket\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.765958 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.765864 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-token\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.765958 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.765890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcb7q\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-kube-api-access-bcb7q\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.765958 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.765922 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-podinfo\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.765958 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.765951 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-socket\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.765972 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-data\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istiod-ca-cert\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766044 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-envoy\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766141 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766076 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-certs\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766345 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766207 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-credential-socket\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766403 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766367 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-socket\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766457 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766424 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-data\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766497 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766471 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-certs\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.766742 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.766724 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istiod-ca-cert\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.768273 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.768245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-envoy\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.768370 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.768359 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-podinfo\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.773935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.773916 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcb7q\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-kube-api-access-bcb7q\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.774127 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.774108 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-token\") pod \"e2e-shared-a-42e123-openshift-default-74759b679-ldssp\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:47.859357 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:47.859297 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:48.033038 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.033014 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp"] Jul 20 16:52:48.036087 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:52:48.036062 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa321cd_3a2e_4c89_b8a0_9e23dbcb7178.slice/crio-44d015b200bfa539b300f7ee29a251f2bba3f3ac4c7f3e628fdff0db57c698ec WatchSource:0}: Error finding container 44d015b200bfa539b300f7ee29a251f2bba3f3ac4c7f3e628fdff0db57c698ec: Status 404 returned error can't find the container with id 44d015b200bfa539b300f7ee29a251f2bba3f3ac4c7f3e628fdff0db57c698ec Jul 20 16:52:48.038351 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.038321 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:52:48.038428 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.038381 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:52:48.038428 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.038415 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:52:48.975161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.975125 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" event={"ID":"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178","Type":"ContainerStarted","Data":"1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a"} Jul 20 16:52:48.975524 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.975176 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" event={"ID":"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178","Type":"ContainerStarted","Data":"44d015b200bfa539b300f7ee29a251f2bba3f3ac4c7f3e628fdff0db57c698ec"} Jul 20 16:52:48.996798 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:48.996748 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" podStartSLOduration=1.99673385 podStartE2EDuration="1.99673385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:48.994168073 +0000 UTC m=+2317.199313813" watchObservedRunningTime="2026-07-20 16:52:48.99673385 +0000 UTC m=+2317.201879591" Jul 20 16:52:49.859774 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:49.859745 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:49.864387 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:49.864362 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:49.979468 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:49.979438 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:52:49.980588 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:52:49.980569 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:53:06.020829 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:06.020794 2582 patch_prober.go:28] interesting pod/e2e-shared-a-42e123-openshift-default-74759b679-ldssp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 16:53:06.023877 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:06.020928 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 16:53:07.074025 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:07.073993 2582 generic.go:356] "Generic (PLEG): container finished" podID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerID="1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a" exitCode=0 Jul 20 16:53:07.074382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:07.074055 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" event={"ID":"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178","Type":"ContainerDied","Data":"1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a"} Jul 20 16:53:07.074382 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:07.074380 2582 scope.go:117] "RemoveContainer" containerID="1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a" Jul 20 16:53:07.075330 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:07.075303 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:53:07.075392 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:07.075379 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:53:07.075442 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:07.075430 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 20 16:53:08.080577 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:08.080536 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" event={"ID":"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178","Type":"ContainerStarted","Data":"d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35"} Jul 20 16:53:08.860056 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:08.860016 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:53:08.860249 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:08.860223 2582 patch_prober.go:28] interesting pod/e2e-shared-a-42e123-openshift-default-74759b679-ldssp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 20 16:53:08.860341 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:08.860315 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 20 16:53:09.860340 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:09.860306 2582 patch_prober.go:28] interesting pod/e2e-shared-a-42e123-openshift-default-74759b679-ldssp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 20 16:53:09.860753 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:09.860395 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 20 16:53:10.860545 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:10.860507 2582 patch_prober.go:28] interesting pod/e2e-shared-a-42e123-openshift-default-74759b679-ldssp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 20 16:53:10.860977 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:10.860582 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 20 16:53:11.863182 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:11.863150 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:53:11.863592 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:11.863426 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:53:11.864208 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:53:11.864190 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:56:08.819522 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.819490 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b"] Jul 20 16:56:08.819946 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.819882 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" Jul 20 16:56:08.819946 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.819893 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="39336b8f-895d-4601-a78f-ec2bf8627da3" containerName="istio-proxy" Jul 20 16:56:08.822959 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.822942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.825725 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.825699 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-42e123\"/\"openshift-service-ca.crt\"" Jul 20 16:56:08.826825 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.826805 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-42e123\"/\"default-dockercfg-lxjxv\"" Jul 20 16:56:08.826932 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.826825 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-42e123\"/\"kube-root-ca.crt\"" Jul 20 16:56:08.826932 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.826854 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-42e123\"/\"sub-test-model-42e123-kserve-self-signed-certs\"" Jul 20 16:56:08.831291 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.831267 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b"] Jul 20 16:56:08.881059 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.881016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tmp-dir\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.881264 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.881080 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-home\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.881264 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.881129 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-dshm\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.881264 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.881195 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tls-certs\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.881264 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.881254 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlt57\" (UniqueName: \"kubernetes.io/projected/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-kube-api-access-dlt57\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.881465 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.881309 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-model-cache\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.981860 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.981832 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-model-cache\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982058 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.981874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tmp-dir\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982058 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.981917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-home\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982058 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.981935 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-dshm\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982058 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.981992 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tls-certs\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982058 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.982049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlt57\" (UniqueName: \"kubernetes.io/projected/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-kube-api-access-dlt57\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.982263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-model-cache\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982335 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.982297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-home\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.982406 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.982339 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tmp-dir\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.984329 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.984297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-dshm\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.984613 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.984593 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tls-certs\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:08.990824 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:08.990805 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlt57\" (UniqueName: \"kubernetes.io/projected/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-kube-api-access-dlt57\") pod \"sub-test-model-42e123-kserve-57c78f949-sfs7b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:09.132545 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:09.132463 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:09.262583 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:09.262556 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b"] Jul 20 16:56:09.264273 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:56:09.264243 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a9fca7c_7cea_42af_a35f_8a87b7f75e9b.slice/crio-2e7e2fa513de444eda9f24216a86b220eab24a994e438c84e55bfdbc0f2c2ddd WatchSource:0}: Error finding container 2e7e2fa513de444eda9f24216a86b220eab24a994e438c84e55bfdbc0f2c2ddd: Status 404 returned error can't find the container with id 2e7e2fa513de444eda9f24216a86b220eab24a994e438c84e55bfdbc0f2c2ddd Jul 20 16:56:09.909296 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:09.909257 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" event={"ID":"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b","Type":"ContainerStarted","Data":"d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766"} Jul 20 16:56:09.909296 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:09.909301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" event={"ID":"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b","Type":"ContainerStarted","Data":"2e7e2fa513de444eda9f24216a86b220eab24a994e438c84e55bfdbc0f2c2ddd"} Jul 20 16:56:09.926938 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:09.926897 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" podStartSLOduration=1.926876523 podStartE2EDuration="1.926876523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:56:09.925249804 +0000 UTC m=+2518.130395546" watchObservedRunningTime="2026-07-20 16:56:09.926876523 +0000 UTC m=+2518.132022265" Jul 20 16:56:19.133069 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:19.133038 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:19.133540 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:19.133082 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:19.145441 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:19.145420 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:56:19.967870 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:56:19.967845 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:57:30.527930 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:30.527899 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b"] Jul 20 16:57:45.547840 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.547744 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" podUID="3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" containerName="main" containerID="cri-o://d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766" gracePeriod=45 Jul 20 16:57:45.795466 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.795445 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:57:45.832886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.832809 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-dshm\") pod \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " Jul 20 16:57:45.832886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.832841 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tls-certs\") pod \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " Jul 20 16:57:45.833083 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.832909 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-model-cache\") pod \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " Jul 20 16:57:45.833083 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.832937 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-home\") pod \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " Jul 20 16:57:45.833083 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.832963 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlt57\" (UniqueName: \"kubernetes.io/projected/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-kube-api-access-dlt57\") pod \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " Jul 20 16:57:45.833083 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.833007 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tmp-dir\") pod \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\" (UID: \"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b\") " Jul 20 16:57:45.833305 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.833173 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" (UID: "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:57:45.833305 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.833190 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-home" (OuterVolumeSpecName: "home") pod "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" (UID: "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:57:45.833305 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.833297 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-model-cache\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:57:45.833474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.833316 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-home\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:57:45.833474 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.833408 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" (UID: "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:57:45.835193 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.835162 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-kube-api-access-dlt57" (OuterVolumeSpecName: "kube-api-access-dlt57") pod "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" (UID: "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b"). InnerVolumeSpecName "kube-api-access-dlt57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:57:45.835362 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.835347 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-dshm" (OuterVolumeSpecName: "dshm") pod "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" (UID: "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:57:45.835415 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.835397 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" (UID: "3a9fca7c-7cea-42af-a35f-8a87b7f75e9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:57:45.934408 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.934382 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlt57\" (UniqueName: \"kubernetes.io/projected/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-kube-api-access-dlt57\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:57:45.934408 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.934407 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tmp-dir\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:57:45.934559 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.934418 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-dshm\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:57:45.934559 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:45.934426 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b-tls-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:57:46.379300 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.379261 2582 generic.go:356] "Generic (PLEG): container finished" podID="3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" containerID="d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766" exitCode=0 Jul 20 16:57:46.379463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.379306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" event={"ID":"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b","Type":"ContainerDied","Data":"d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766"} Jul 20 16:57:46.379463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.379308 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" Jul 20 16:57:46.379463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.379332 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b" event={"ID":"3a9fca7c-7cea-42af-a35f-8a87b7f75e9b","Type":"ContainerDied","Data":"2e7e2fa513de444eda9f24216a86b220eab24a994e438c84e55bfdbc0f2c2ddd"} Jul 20 16:57:46.379463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.379349 2582 scope.go:117] "RemoveContainer" containerID="d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766" Jul 20 16:57:46.387797 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.387782 2582 scope.go:117] "RemoveContainer" containerID="d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766" Jul 20 16:57:46.388030 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:57:46.388012 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766\": container with ID starting with d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766 not found: ID does not exist" containerID="d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766" Jul 20 16:57:46.388074 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.388037 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766"} err="failed to get container status \"d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766\": rpc error: code = NotFound desc = could not find container \"d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766\": container with ID starting with d1b2a8915ef892cae7b20ed9affbd0c6dc194adc8c6affb967b7c40f78329766 not found: ID does not exist" Jul 20 16:57:46.396814 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.396789 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b"] Jul 20 16:57:46.402921 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:46.402899 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-42e123/sub-test-model-42e123-kserve-57c78f949-sfs7b"] Jul 20 16:57:48.328592 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:57:48.328548 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" path="/var/lib/kubelet/pods/3a9fca7c-7cea-42af-a35f-8a87b7f75e9b/volumes" Jul 20 16:58:08.006934 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.006897 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q"] Jul 20 16:58:08.009151 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.007488 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" containerName="main" Jul 20 16:58:08.009151 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.007508 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" containerName="main" Jul 20 16:58:08.009151 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.007607 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a9fca7c-7cea-42af-a35f-8a87b7f75e9b" containerName="main" Jul 20 16:58:08.010730 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.010707 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.014869 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.014844 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c5c3fb\"/\"rate-test-model-c5c3fb-kserve-self-signed-certs\"" Jul 20 16:58:08.015057 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.014862 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c5c3fb\"/\"openshift-service-ca.crt\"" Jul 20 16:58:08.015172 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.014916 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c5c3fb\"/\"default-dockercfg-nk5nc\"" Jul 20 16:58:08.015172 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.014945 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c5c3fb\"/\"kube-root-ca.crt\"" Jul 20 16:58:08.021631 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.021589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q"] Jul 20 16:58:08.142938 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.142903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-home\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.142938 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.142938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-dshm\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.143161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.142965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50173944-c4cd-46f4-a82e-df2561d213e7-tls-certs\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.143161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.143113 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk6pq\" (UniqueName: \"kubernetes.io/projected/50173944-c4cd-46f4-a82e-df2561d213e7-kube-api-access-bk6pq\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.143161 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.143151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-tmp-dir\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.143321 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.143276 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-model-cache\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244448 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244410 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk6pq\" (UniqueName: \"kubernetes.io/projected/50173944-c4cd-46f4-a82e-df2561d213e7-kube-api-access-bk6pq\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244448 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-tmp-dir\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244495 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-model-cache\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-home\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-dshm\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244757 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50173944-c4cd-46f4-a82e-df2561d213e7-tls-certs\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.244972 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244950 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-model-cache\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.245014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.244960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-tmp-dir\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.245054 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.245021 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-home\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.247046 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.247027 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-dshm\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.247285 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.247264 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50173944-c4cd-46f4-a82e-df2561d213e7-tls-certs\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.252834 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.252812 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk6pq\" (UniqueName: \"kubernetes.io/projected/50173944-c4cd-46f4-a82e-df2561d213e7-kube-api-access-bk6pq\") pod \"rate-test-model-c5c3fb-kserve-558665764c-p2g5q\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.321383 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.321306 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:08.454473 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.454450 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q"] Jul 20 16:58:08.456349 ip-10-0-131-211 kubenswrapper[2582]: W0720 16:58:08.456322 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50173944_c4cd_46f4_a82e_df2561d213e7.slice/crio-819912b6374508cf58255fc13ff59ebc1033b42d110214d0ce2f67b29573ad5d WatchSource:0}: Error finding container 819912b6374508cf58255fc13ff59ebc1033b42d110214d0ce2f67b29573ad5d: Status 404 returned error can't find the container with id 819912b6374508cf58255fc13ff59ebc1033b42d110214d0ce2f67b29573ad5d Jul 20 16:58:08.481452 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:08.481428 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" event={"ID":"50173944-c4cd-46f4-a82e-df2561d213e7","Type":"ContainerStarted","Data":"819912b6374508cf58255fc13ff59ebc1033b42d110214d0ce2f67b29573ad5d"} Jul 20 16:58:09.487692 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:09.487652 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" event={"ID":"50173944-c4cd-46f4-a82e-df2561d213e7","Type":"ContainerStarted","Data":"6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d"} Jul 20 16:58:09.506173 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:09.506124 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" podStartSLOduration=2.506109221 podStartE2EDuration="2.506109221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:58:09.50515886 +0000 UTC m=+2637.710304613" watchObservedRunningTime="2026-07-20 16:58:09.506109221 +0000 UTC m=+2637.711254963" Jul 20 16:58:18.328280 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:18.328249 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:18.328702 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:18.328291 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:18.335014 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:18.334994 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:18.545475 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:18.545441 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:58:53.011853 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:58:53.011819 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q"] Jul 20 16:59:08.035383 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.035328 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" podUID="50173944-c4cd-46f4-a82e-df2561d213e7" containerName="main" containerID="cri-o://6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d" gracePeriod=45 Jul 20 16:59:08.282523 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.282498 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:59:08.308261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308192 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-model-cache\") pod \"50173944-c4cd-46f4-a82e-df2561d213e7\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " Jul 20 16:59:08.308261 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308246 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-dshm\") pod \"50173944-c4cd-46f4-a82e-df2561d213e7\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " Jul 20 16:59:08.308438 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308286 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50173944-c4cd-46f4-a82e-df2561d213e7-tls-certs\") pod \"50173944-c4cd-46f4-a82e-df2561d213e7\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " Jul 20 16:59:08.308438 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308332 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-home\") pod \"50173944-c4cd-46f4-a82e-df2561d213e7\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " Jul 20 16:59:08.308438 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308355 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk6pq\" (UniqueName: \"kubernetes.io/projected/50173944-c4cd-46f4-a82e-df2561d213e7-kube-api-access-bk6pq\") pod \"50173944-c4cd-46f4-a82e-df2561d213e7\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " Jul 20 16:59:08.308438 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308413 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-tmp-dir\") pod \"50173944-c4cd-46f4-a82e-df2561d213e7\" (UID: \"50173944-c4cd-46f4-a82e-df2561d213e7\") " Jul 20 16:59:08.308661 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308457 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "50173944-c4cd-46f4-a82e-df2561d213e7" (UID: "50173944-c4cd-46f4-a82e-df2561d213e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:08.308720 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308691 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-home" (OuterVolumeSpecName: "home") pod "50173944-c4cd-46f4-a82e-df2561d213e7" (UID: "50173944-c4cd-46f4-a82e-df2561d213e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:08.308906 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308867 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-home\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:08.308906 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308882 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50173944-c4cd-46f4-a82e-df2561d213e7" (UID: "50173944-c4cd-46f4-a82e-df2561d213e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:08.309081 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.308889 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-model-cache\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:08.310770 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.310740 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50173944-c4cd-46f4-a82e-df2561d213e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50173944-c4cd-46f4-a82e-df2561d213e7" (UID: "50173944-c4cd-46f4-a82e-df2561d213e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:59:08.311019 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.310992 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-dshm" (OuterVolumeSpecName: "dshm") pod "50173944-c4cd-46f4-a82e-df2561d213e7" (UID: "50173944-c4cd-46f4-a82e-df2561d213e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:08.311102 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.310991 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50173944-c4cd-46f4-a82e-df2561d213e7-kube-api-access-bk6pq" (OuterVolumeSpecName: "kube-api-access-bk6pq") pod "50173944-c4cd-46f4-a82e-df2561d213e7" (UID: "50173944-c4cd-46f4-a82e-df2561d213e7"). InnerVolumeSpecName "kube-api-access-bk6pq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:59:08.409771 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.409743 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50173944-c4cd-46f4-a82e-df2561d213e7-tls-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:08.409771 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.409766 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk6pq\" (UniqueName: \"kubernetes.io/projected/50173944-c4cd-46f4-a82e-df2561d213e7-kube-api-access-bk6pq\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:08.409771 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.409776 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-tmp-dir\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:08.409955 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.409785 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50173944-c4cd-46f4-a82e-df2561d213e7-dshm\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:08.766294 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.766256 2582 generic.go:356] "Generic (PLEG): container finished" podID="50173944-c4cd-46f4-a82e-df2561d213e7" containerID="6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d" exitCode=0 Jul 20 16:59:08.766470 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.766305 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" Jul 20 16:59:08.766470 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.766335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" event={"ID":"50173944-c4cd-46f4-a82e-df2561d213e7","Type":"ContainerDied","Data":"6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d"} Jul 20 16:59:08.766470 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.766381 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q" event={"ID":"50173944-c4cd-46f4-a82e-df2561d213e7","Type":"ContainerDied","Data":"819912b6374508cf58255fc13ff59ebc1033b42d110214d0ce2f67b29573ad5d"} Jul 20 16:59:08.766470 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.766398 2582 scope.go:117] "RemoveContainer" containerID="6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d" Jul 20 16:59:08.774974 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.774956 2582 scope.go:117] "RemoveContainer" containerID="6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d" Jul 20 16:59:08.775256 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:59:08.775237 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d\": container with ID starting with 6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d not found: ID does not exist" containerID="6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d" Jul 20 16:59:08.775302 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.775265 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d"} err="failed to get container status \"6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d\": rpc error: code = NotFound desc = could not find container \"6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d\": container with ID starting with 6b19963803f3b8638d53c2710058284b9d4335d7ced45a7399b2a037493a431d not found: ID does not exist" Jul 20 16:59:08.783665 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.783639 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q"] Jul 20 16:59:08.787339 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:08.787317 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c5c3fb/rate-test-model-c5c3fb-kserve-558665764c-p2g5q"] Jul 20 16:59:10.327707 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:10.327671 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50173944-c4cd-46f4-a82e-df2561d213e7" path="/var/lib/kubelet/pods/50173944-c4cd-46f4-a82e-df2561d213e7/volumes" Jul 20 16:59:34.210935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:34.210897 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp"] Jul 20 16:59:34.211452 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:34.211210 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" containerID="cri-o://d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35" gracePeriod=30 Jul 20 16:59:39.464313 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.464290 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:59:39.600181 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600102 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-socket\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600181 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600145 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-token\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600181 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600168 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-podinfo\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600188 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcb7q\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-kube-api-access-bcb7q\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600241 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-certs\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600279 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-credential-socket\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600309 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-data\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600329 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istiod-ca-cert\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600360 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-envoy\") pod \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\" (UID: \"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178\") " Jul 20 16:59:39.600463 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600392 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:39.600855 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600647 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.600855 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600660 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:39.600855 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600676 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:39.600855 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.600745 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:59:39.601259 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.601236 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:39.602557 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.602536 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-kube-api-access-bcb7q" (OuterVolumeSpecName: "kube-api-access-bcb7q") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "kube-api-access-bcb7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:59:39.602657 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.602552 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:59:39.602750 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.602730 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:59:39.602828 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.602803 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" (UID: "1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:59:39.701461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701429 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-token\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701456 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-podinfo\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701461 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701466 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcb7q\" (UniqueName: \"kubernetes.io/projected/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-kube-api-access-bcb7q\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701692 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701477 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-workload-certs\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701692 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701486 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-credential-socket\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701692 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701495 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-data\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701692 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701503 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istiod-ca-cert\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.701692 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.701511 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178-istio-envoy\") on node \"ip-10-0-131-211.ec2.internal\" DevicePath \"\"" Jul 20 16:59:39.912644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.912525 2582 generic.go:356] "Generic (PLEG): container finished" podID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerID="d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35" exitCode=0 Jul 20 16:59:39.912644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.912598 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" Jul 20 16:59:39.912644 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.912612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" event={"ID":"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178","Type":"ContainerDied","Data":"d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35"} Jul 20 16:59:39.912886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.912707 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp" event={"ID":"1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178","Type":"ContainerDied","Data":"44d015b200bfa539b300f7ee29a251f2bba3f3ac4c7f3e628fdff0db57c698ec"} Jul 20 16:59:39.912886 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.912730 2582 scope.go:117] "RemoveContainer" containerID="d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35" Jul 20 16:59:39.922106 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.922090 2582 scope.go:117] "RemoveContainer" containerID="1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a" Jul 20 16:59:39.929582 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.929563 2582 scope.go:117] "RemoveContainer" containerID="d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35" Jul 20 16:59:39.929855 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:59:39.929836 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35\": container with ID starting with d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35 not found: ID does not exist" containerID="d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35" Jul 20 16:59:39.929927 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.929864 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35"} err="failed to get container status \"d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35\": rpc error: code = NotFound desc = could not find container \"d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35\": container with ID starting with d81a0d056c9f0b28e5224acfb15ff6f0c62901fdca65af79db834169b31d4b35 not found: ID does not exist" Jul 20 16:59:39.929927 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.929881 2582 scope.go:117] "RemoveContainer" containerID="1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a" Jul 20 16:59:39.930101 ip-10-0-131-211 kubenswrapper[2582]: E0720 16:59:39.930084 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a\": container with ID starting with 1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a not found: ID does not exist" containerID="1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a" Jul 20 16:59:39.930139 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.930105 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a"} err="failed to get container status \"1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a\": rpc error: code = NotFound desc = could not find container \"1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a\": container with ID starting with 1cc56d36c74be19e15ac83b14e267f9c7db2197bd4b927c1cce538f3fcb17d1a not found: ID does not exist" Jul 20 16:59:39.934935 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.934915 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp"] Jul 20 16:59:39.943189 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:39.943169 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-42e123-openshift-default-74759b679-ldssp"] Jul 20 16:59:40.327862 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:40.327819 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" path="/var/lib/kubelet/pods/1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178/volumes" Jul 20 16:59:56.984960 ip-10-0-131-211 kubenswrapper[2582]: I0720 16:59:56.984927 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4658cbf5-6nbvj_88bfe08d-42c1-43af-9333-4a75539fb80e/authorino/0.log" Jul 20 17:00:02.376091 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:02.376059 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67b54448d8-q2b7v_95d50ae2-17f6-41ac-a00b-cf1b6fa17260/manager/0.log" Jul 20 17:00:02.824419 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:02.824387 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bbb748554-9kcjv_9528ffd8-aac4-40e8-b261-35a585bbf31d/copy-manifests/0.log" Jul 20 17:00:02.836887 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:02.836855 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bbb748554-9kcjv_9528ffd8-aac4-40e8-b261-35a585bbf31d/manager/0.log" Jul 20 17:00:05.015068 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:05.015033 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4658cbf5-6nbvj_88bfe08d-42c1-43af-9333-4a75539fb80e/authorino/0.log" Jul 20 17:00:05.348275 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:05.348202 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-czhkk_3c00db93-d944-4b47-aaf8-abf47985b9f7/kuadrant-console-plugin/0.log" Jul 20 17:00:05.583956 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:05.583924 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bbsfd_08900eb9-f367-4b1a-b9c8-ba9c79b7c952/manager/0.log" Jul 20 17:00:06.160603 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:06.160577 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46slp7_fbbc242c-5106-41e2-b37d-1964f0d9c0da/istio-proxy/0.log" Jul 20 17:00:06.390798 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:06.390773 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4f98b7bc-4krfw_4456216b-ebab-4f9b-b312-9bd03055ab3a/kube-auth-proxy/0.log" Jul 20 17:00:06.855156 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:06.855124 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wkpz9_bf527d47-1635-4c85-9b2a-2040d021bb18/payload-processing/0.log" Jul 20 17:00:07.536427 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:07.536396 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9t7mk_40dfa8ce-5eb2-4cf1-8fa9-254aa153058c/main/0.log" Jul 20 17:00:07.863812 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:07.863730 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9xh82_3a9de5a5-6f02-4577-9088-29c38f901d58/main/0.log" Jul 20 17:00:16.448448 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:16.448419 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-szff5_6dade5e1-3c11-4fd3-8c58-3dad3bf4f1a4/global-pull-secret-syncer/0.log" Jul 20 17:00:16.576350 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:16.576322 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rlr9z_3081a2dc-7efc-4631-ab62-cc41fa045f67/konnectivity-agent/0.log" Jul 20 17:00:16.598516 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:16.598494 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-211.ec2.internal_ad93a0f02d30ff499207190d61ffb0a2/haproxy/0.log" Jul 20 17:00:21.589708 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:21.589671 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4658cbf5-6nbvj_88bfe08d-42c1-43af-9333-4a75539fb80e/authorino/0.log" Jul 20 17:00:21.683200 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:21.683173 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-czhkk_3c00db93-d944-4b47-aaf8-abf47985b9f7/kuadrant-console-plugin/0.log" Jul 20 17:00:21.892847 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:21.892770 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bbsfd_08900eb9-f367-4b1a-b9c8-ba9c79b7c952/manager/0.log" Jul 20 17:00:23.793269 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.793240 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/alertmanager/0.log" Jul 20 17:00:23.817088 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.817067 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/config-reloader/0.log" Jul 20 17:00:23.839743 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.839721 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/kube-rbac-proxy-web/0.log" Jul 20 17:00:23.863200 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.863183 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/kube-rbac-proxy/0.log" Jul 20 17:00:23.885123 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.885098 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/kube-rbac-proxy-metric/0.log" Jul 20 17:00:23.909638 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.909554 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/prom-label-proxy/0.log" Jul 20 17:00:23.934731 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:23.934710 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1033d9d8-d648-4d33-ad6b-4f8b5f086566/init-config-reloader/0.log" Jul 20 17:00:24.180720 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.180689 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g22m4_1e661263-e149-45a6-b34a-fa68e4bd7282/node-exporter/0.log" Jul 20 17:00:24.206631 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.206601 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g22m4_1e661263-e149-45a6-b34a-fa68e4bd7282/kube-rbac-proxy/0.log" Jul 20 17:00:24.238167 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.238140 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g22m4_1e661263-e149-45a6-b34a-fa68e4bd7282/init-textfile/0.log" Jul 20 17:00:24.437495 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.437429 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-2rz9h_1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c/kube-rbac-proxy-main/0.log" Jul 20 17:00:24.464140 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.464117 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-2rz9h_1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c/kube-rbac-proxy-self/0.log" Jul 20 17:00:24.489744 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.489714 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-2rz9h_1ccfa788-fb2e-4d7a-9fbb-66c1c51c8d3c/openshift-state-metrics/0.log" Jul 20 17:00:24.528917 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.528897 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/prometheus/0.log" Jul 20 17:00:24.569140 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.569122 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/config-reloader/0.log" Jul 20 17:00:24.596222 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.596205 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/thanos-sidecar/0.log" Jul 20 17:00:24.619093 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.619052 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/kube-rbac-proxy-web/0.log" Jul 20 17:00:24.644953 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.644935 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/kube-rbac-proxy/0.log" Jul 20 17:00:24.667505 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.667490 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/kube-rbac-proxy-thanos/0.log" Jul 20 17:00:24.690091 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.690036 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e74dca-e0cf-4fb0-96bc-460464a09573/init-config-reloader/0.log" Jul 20 17:00:24.812691 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.812672 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-6xbs4_69a287c4-7154-4f30-8586-b3516793335a/prometheus-operator-admission-webhook/0.log" Jul 20 17:00:24.929595 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.929572 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd8845757-9lkwc_810cf5e9-67a5-46b8-9158-c4eb638b3ef0/thanos-query/0.log" Jul 20 17:00:24.954413 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.954359 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd8845757-9lkwc_810cf5e9-67a5-46b8-9158-c4eb638b3ef0/kube-rbac-proxy-web/0.log" Jul 20 17:00:24.986338 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:24.986321 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd8845757-9lkwc_810cf5e9-67a5-46b8-9158-c4eb638b3ef0/kube-rbac-proxy/0.log" Jul 20 17:00:25.010248 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:25.010227 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd8845757-9lkwc_810cf5e9-67a5-46b8-9158-c4eb638b3ef0/prom-label-proxy/0.log" Jul 20 17:00:25.035638 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:25.035603 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd8845757-9lkwc_810cf5e9-67a5-46b8-9158-c4eb638b3ef0/kube-rbac-proxy-rules/0.log" Jul 20 17:00:25.061881 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:25.061864 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd8845757-9lkwc_810cf5e9-67a5-46b8-9158-c4eb638b3ef0/kube-rbac-proxy-metrics/0.log" Jul 20 17:00:26.309761 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.309714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x"] Jul 20 17:00:26.310153 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310095 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" Jul 20 17:00:26.310153 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310105 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" Jul 20 17:00:26.310153 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310123 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50173944-c4cd-46f4-a82e-df2561d213e7" containerName="main" Jul 20 17:00:26.310153 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310130 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="50173944-c4cd-46f4-a82e-df2561d213e7" containerName="main" Jul 20 17:00:26.310302 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310191 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" Jul 20 17:00:26.310302 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310201 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa321cd-3a2e-4c89-b8a0-9e23dbcb7178" containerName="istio-proxy" Jul 20 17:00:26.310302 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.310208 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="50173944-c4cd-46f4-a82e-df2561d213e7" containerName="main" Jul 20 17:00:26.313480 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.313458 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.316708 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.316687 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6xq7b\"/\"kube-root-ca.crt\"" Jul 20 17:00:26.316708 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.316708 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6xq7b\"/\"default-dockercfg-gflxw\"" Jul 20 17:00:26.317633 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.317598 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6xq7b\"/\"openshift-service-ca.crt\"" Jul 20 17:00:26.329272 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.329251 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x"] Jul 20 17:00:26.415307 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.415275 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-444dp\" (UniqueName: \"kubernetes.io/projected/81c67730-87c0-42f4-99bd-2749701f2418-kube-api-access-444dp\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.415448 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.415327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-sys\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.415448 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.415411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-podres\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.415448 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.415444 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-lib-modules\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.415552 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.415472 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-proc\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.516869 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.516827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-sys\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517030 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.516895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-podres\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517030 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.516917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-lib-modules\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517030 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.516940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-proc\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517030 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.516943 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-sys\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517030 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.516992 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-proc\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517248 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.517037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-444dp\" (UniqueName: \"kubernetes.io/projected/81c67730-87c0-42f4-99bd-2749701f2418-kube-api-access-444dp\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517248 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.517064 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-lib-modules\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.517248 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.517071 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81c67730-87c0-42f4-99bd-2749701f2418-podres\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.526172 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.526147 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-444dp\" (UniqueName: \"kubernetes.io/projected/81c67730-87c0-42f4-99bd-2749701f2418-kube-api-access-444dp\") pod \"perf-node-gather-daemonset-qfs4x\" (UID: \"81c67730-87c0-42f4-99bd-2749701f2418\") " pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.624541 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.624471 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:26.749540 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:26.749509 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x"] Jul 20 17:00:26.752008 ip-10-0-131-211 kubenswrapper[2582]: W0720 17:00:26.751979 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81c67730_87c0_42f4_99bd_2749701f2418.slice/crio-715ad3c5d0ba18fe1474d0795be4bff5575861463a7ee8234394a5b016c0eb27 WatchSource:0}: Error finding container 715ad3c5d0ba18fe1474d0795be4bff5575861463a7ee8234394a5b016c0eb27: Status 404 returned error can't find the container with id 715ad3c5d0ba18fe1474d0795be4bff5575861463a7ee8234394a5b016c0eb27 Jul 20 17:00:27.126250 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:27.126220 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" event={"ID":"81c67730-87c0-42f4-99bd-2749701f2418","Type":"ContainerStarted","Data":"620abc6a48a7f89fb1913c2530ade2f36bb9c559e84b3a98c209028764db4b2b"} Jul 20 17:00:27.126412 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:27.126269 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" event={"ID":"81c67730-87c0-42f4-99bd-2749701f2418","Type":"ContainerStarted","Data":"715ad3c5d0ba18fe1474d0795be4bff5575861463a7ee8234394a5b016c0eb27"} Jul 20 17:00:27.126412 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:27.126297 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:27.143346 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:27.143301 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" podStartSLOduration=1.143287546 podStartE2EDuration="1.143287546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:00:27.142400714 +0000 UTC m=+2775.347546457" watchObservedRunningTime="2026-07-20 17:00:27.143287546 +0000 UTC m=+2775.348433361" Jul 20 17:00:27.313946 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:27.313923 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75b5ffb47c-whthc_3668c7b0-ffc6-4ef1-93b0-f4d5d6d76965/console/0.log" Jul 20 17:00:29.155105 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:29.155077 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkt8q_c4e6ee43-be51-4ce7-9617-25dafef1add4/dns/0.log" Jul 20 17:00:29.178585 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:29.178551 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkt8q_c4e6ee43-be51-4ce7-9617-25dafef1add4/kube-rbac-proxy/0.log" Jul 20 17:00:29.228183 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:29.228152 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hlwlt_21384c1f-0989-4be9-ad26-4586bf5e1511/dns-node-resolver/0.log" Jul 20 17:00:29.902248 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:29.902213 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-szh9h_081af134-9948-49cf-9b25-a05824539331/node-ca/0.log" Jul 20 17:00:30.860166 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:30.860135 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46slp7_fbbc242c-5106-41e2-b37d-1964f0d9c0da/istio-proxy/0.log" Jul 20 17:00:31.124064 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:31.123977 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4f98b7bc-4krfw_4456216b-ebab-4f9b-b312-9bd03055ab3a/kube-auth-proxy/0.log" Jul 20 17:00:31.277069 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:31.277036 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wkpz9_bf527d47-1635-4c85-9b2a-2040d021bb18/payload-processing/0.log" Jul 20 17:00:31.975584 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:31.975556 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ff5cq_b6fde27d-e433-493b-ab11-b73ccf046951/serve-healthcheck-canary/0.log" Jul 20 17:00:32.704162 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:32.704135 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4bdxf_a4851e3b-efcd-4291-8c66-61efff445cc6/kube-rbac-proxy/0.log" Jul 20 17:00:32.725874 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:32.725847 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4bdxf_a4851e3b-efcd-4291-8c66-61efff445cc6/exporter/0.log" Jul 20 17:00:32.746949 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:32.746912 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4bdxf_a4851e3b-efcd-4291-8c66-61efff445cc6/extractor/0.log" Jul 20 17:00:33.140721 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:33.140644 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6xq7b/perf-node-gather-daemonset-qfs4x" Jul 20 17:00:35.311611 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:35.311571 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67b54448d8-q2b7v_95d50ae2-17f6-41ac-a00b-cf1b6fa17260/manager/0.log" Jul 20 17:00:35.543679 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:35.543647 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bbb748554-9kcjv_9528ffd8-aac4-40e8-b261-35a585bbf31d/manager/0.log" Jul 20 17:00:35.563428 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:35.563408 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bbb748554-9kcjv_9528ffd8-aac4-40e8-b261-35a585bbf31d/copy-manifests/0.log" Jul 20 17:00:36.631976 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:36.631897 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46slp7_fbbc242c-5106-41e2-b37d-1964f0d9c0da/istio-proxy/0.log" Jul 20 17:00:36.862369 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:36.862344 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4f98b7bc-4krfw_4456216b-ebab-4f9b-b312-9bd03055ab3a/kube-auth-proxy/0.log" Jul 20 17:00:36.956261 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:36.956227 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7797c4479d-f9kmr_5b4216d7-9f37-4525-9399-d1e6fcf0c98f/manager/0.log" Jul 20 17:00:36.961435 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:36.961414 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wkpz9_bf527d47-1635-4c85-9b2a-2040d021bb18/payload-processing/0.log" Jul 20 17:00:38.273962 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:38.273933 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46slp7_fbbc242c-5106-41e2-b37d-1964f0d9c0da/istio-proxy/0.log" Jul 20 17:00:38.551810 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:38.551705 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4f98b7bc-4krfw_4456216b-ebab-4f9b-b312-9bd03055ab3a/kube-auth-proxy/0.log" Jul 20 17:00:38.663339 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:38.663310 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wkpz9_bf527d47-1635-4c85-9b2a-2040d021bb18/payload-processing/0.log" Jul 20 17:00:43.439544 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.439514 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/kube-multus-additional-cni-plugins/0.log" Jul 20 17:00:43.461778 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.461750 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/egress-router-binary-copy/0.log" Jul 20 17:00:43.483611 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.483587 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/cni-plugins/0.log" Jul 20 17:00:43.504857 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.504830 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/bond-cni-plugin/0.log" Jul 20 17:00:43.527455 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.527435 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/routeoverride-cni/0.log" Jul 20 17:00:43.548782 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.548757 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/whereabouts-cni-bincopy/0.log" Jul 20 17:00:43.573420 ip-10-0-131-211 kubenswrapper[2582]: I0720 17:00:43.573395 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4x6m8_d89302d9-1985-400e-a259-c9fcd79c0e77/whereabouts-cni/0.log"