Jul 07 17:07:26.439264 ip-10-0-133-16 systemd[1]: Starting Kubernetes Kubelet... Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.881536 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888260 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888272 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888276 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888279 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888282 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888285 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888289 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:07:26.934451 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888291 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888294 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888297 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888300 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888303 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888307 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888310 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888312 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888315 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888318 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888320 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888323 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888327 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888329 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888332 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888335 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888338 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888341 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888344 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888346 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:07:26.937457 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888349 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888352 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888355 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888357 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888360 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888362 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888367 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888370 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888380 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888383 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888385 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888388 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888390 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888393 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888396 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888398 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888401 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888405 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888407 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:07:26.941126 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888410 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888412 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888415 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888417 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888420 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888422 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888425 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888427 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888430 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888433 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888436 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888438 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888441 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888444 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888447 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888449 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888452 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888455 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888458 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888460 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:07:26.943981 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888462 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888465 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888468 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888471 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888474 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888477 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888480 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888483 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888486 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888488 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888491 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888494 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888497 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888500 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888502 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888505 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888507 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888510 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888513 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888515 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:07:26.947662 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888518 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888520 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888523 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888525 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888529 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888533 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888536 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888540 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888544 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888547 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888550 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888553 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.888556 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.891928 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.897341 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899572 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899583 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899590 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:07:26.950517 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899595 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899601 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899605 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899610 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899616 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899621 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899625 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899629 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899633 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.899638 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902424 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902435 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902441 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902446 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902451 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902455 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902459 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902463 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902467 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:07:26.954480 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902473 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902478 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902482 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.902486 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904743 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904752 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904757 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904762 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904766 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904771 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904776 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904781 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904786 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904791 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904796 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904800 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904804 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.904808 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908008 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908016 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908025 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:07:26.957866 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908029 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908036 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908040 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908044 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908048 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908052 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908057 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.908061 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910313 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910320 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910324 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910329 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910334 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910338 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910347 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910351 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910355 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910360 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910366 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910371 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:07:26.961604 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910376 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910380 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910384 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910389 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910393 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910397 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910404 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910411 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910416 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910421 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910425 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.910429 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913234 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913242 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913247 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913251 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913257 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913262 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913266 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:07:26.964619 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913272 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913276 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913281 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913287 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913291 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913295 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913299 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.913303 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915514 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915522 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915526 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915530 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915535 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915540 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.915544 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926461 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926475 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926479 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926482 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:07:26.968743 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926486 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926490 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926494 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926508 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926511 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926514 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926517 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926521 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926523 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926526 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926529 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926532 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926535 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926538 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926541 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926543 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926546 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926548 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926551 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926553 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:07:26.972179 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926555 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926559 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926562 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926564 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926567 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926570 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926573 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926575 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926578 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926581 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926585 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926589 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926591 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926594 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926597 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926599 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926602 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926605 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926608 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926611 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:07:26.977173 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926614 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926617 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926620 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926623 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926625 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926628 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926631 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926634 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926637 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926639 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926642 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926646 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926649 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926652 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926658 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926661 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926663 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926666 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926668 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:07:26.980306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926671 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926674 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926677 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926680 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926682 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926685 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926687 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926690 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926693 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926696 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926698 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926701 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926703 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926706 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926709 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926712 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926715 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926717 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926720 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:07:26.984154 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926725 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926727 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926730 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926732 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926734 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926737 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926740 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926742 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926746 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926748 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926751 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926754 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926756 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926759 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926761 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926764 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.926767 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926844 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926852 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926859 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926864 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 17:07:26.986975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926868 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926872 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926876 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926881 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926884 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926887 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926891 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926894 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926897 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926900 2582 flags.go:64] FLAG: --cgroup-root="" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926904 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926906 2582 flags.go:64] FLAG: --client-ca-file="" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926911 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926914 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926919 2582 flags.go:64] FLAG: --cluster-domain="" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926922 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926925 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926928 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926946 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926952 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926955 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926958 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926962 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926965 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 07 17:07:26.990515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926968 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926971 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926974 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926977 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926981 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926984 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926988 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926991 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926994 2582 flags.go:64] FLAG: --enable-server="true" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.926997 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927002 2582 flags.go:64] FLAG: --event-burst="100" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927005 2582 flags.go:64] FLAG: --event-qps="50" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927008 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927011 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927014 2582 flags.go:64] FLAG: --eviction-hard="" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927018 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927022 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927025 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927028 2582 flags.go:64] FLAG: --eviction-soft="" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927031 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927034 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927037 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927041 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927044 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927047 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 17:07:26.993443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927050 2582 flags.go:64] FLAG: --feature-gates="" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927054 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927059 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927062 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927065 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927069 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927072 2582 flags.go:64] FLAG: --help="false" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927074 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-133-16.ec2.internal" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927078 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927081 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927084 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927087 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927090 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927093 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927096 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927099 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927102 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927106 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927110 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927112 2582 flags.go:64] FLAG: --kube-reserved="" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927115 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927118 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927121 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927124 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 17:07:26.997339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927127 2582 flags.go:64] FLAG: --lock-file="" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927129 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927132 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927135 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927141 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927144 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927147 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927150 2582 flags.go:64] FLAG: --logging-format="text" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927153 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927156 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927160 2582 flags.go:64] FLAG: --manifest-url="" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927163 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927169 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927171 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927176 2582 flags.go:64] FLAG: --max-pods="110" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927179 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927182 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927185 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927188 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927191 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927194 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927197 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927204 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927207 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927210 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 17:07:27.031719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927214 2582 flags.go:64] FLAG: --pod-cidr="" Jul 07 17:07:27.001110 ip-10-0-133-16 systemd[1]: Started Kubernetes Kubelet. Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927217 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927223 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927226 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927229 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927232 2582 flags.go:64] FLAG: --port="10250" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927235 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927238 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01d6af71d04836a90" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927241 2582 flags.go:64] FLAG: --qos-reserved="" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927244 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927247 2582 flags.go:64] FLAG: --register-node="true" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927250 2582 flags.go:64] FLAG: --register-with-taints="" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927254 2582 flags.go:64] FLAG: --registry-burst="10" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927257 2582 flags.go:64] FLAG: --registry-qps="5" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927260 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927262 2582 flags.go:64] FLAG: --reserved-memory="" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927266 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927269 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927273 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927276 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927279 2582 flags.go:64] FLAG: --runonce="false" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927282 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927285 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927288 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927291 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 17:07:27.050281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927294 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927296 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927299 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927302 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927305 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927308 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927311 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927314 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927318 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927320 2582 flags.go:64] FLAG: --system-cgroups="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927323 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927329 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927332 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927334 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927338 2582 flags.go:64] FLAG: --tls-min-version="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927341 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927344 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927347 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927350 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927353 2582 flags.go:64] FLAG: --v="2" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927356 2582 flags.go:64] FLAG: --version="false" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927361 2582 flags.go:64] FLAG: --vmodule="" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927365 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.927368 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927495 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:07:27.074985 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927500 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927504 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927507 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927510 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927513 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927516 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927519 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927522 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927525 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927527 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927530 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927533 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927536 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927539 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927545 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927548 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927552 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927555 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927558 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:07:27.076529 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927561 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927564 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927566 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927569 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927573 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927575 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927578 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927581 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927583 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927586 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927593 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927596 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927599 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927603 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927605 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927608 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927610 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927613 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927615 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927618 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:07:27.077534 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927621 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927623 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927626 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927628 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927631 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927634 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927637 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927641 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927643 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927646 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927648 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927651 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927654 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927657 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927659 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927662 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927665 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927668 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927672 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927675 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:07:27.078568 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927678 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927681 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927683 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927686 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927689 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927693 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927696 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927698 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927700 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927703 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927706 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927708 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927711 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927713 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927716 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927718 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927721 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927723 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927726 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:07:27.079609 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927729 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927732 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927734 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927737 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927740 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927742 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927744 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927747 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927750 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927752 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927755 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927757 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927760 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927762 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927765 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927767 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927770 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927773 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927777 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:07:27.080637 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.927780 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.928477 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.935631 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.935648 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.935669 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.935676 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935737 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935744 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935748 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935751 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935754 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935756 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935759 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935762 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:07:27.081652 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935765 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935767 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935770 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935773 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935776 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935779 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935782 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935785 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935788 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935790 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935793 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935795 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935798 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935801 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935803 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935806 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935808 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935812 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935814 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935816 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:07:27.082642 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935822 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935826 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935829 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935832 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935835 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935838 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935840 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935843 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935846 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935849 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935851 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935854 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935856 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935859 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935861 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935864 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935868 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935872 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935875 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:07:27.083657 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935878 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935881 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935883 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935886 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935888 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935891 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935893 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935896 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935901 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935903 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935906 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935909 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935912 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935916 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935919 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935922 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935925 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935928 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935947 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935952 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:07:27.084667 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935955 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935959 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935961 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935964 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935966 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935969 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935971 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935974 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935976 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935979 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935981 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935984 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935987 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935989 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935991 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935994 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935996 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.935999 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936001 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:07:27.085684 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936004 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936008 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936012 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936015 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936019 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936022 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936026 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936029 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936032 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936035 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936038 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936040 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936043 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.936048 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:07:27.086700 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936168 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936174 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936177 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936180 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936183 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936186 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936189 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936191 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936194 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936197 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936200 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936202 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936205 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936208 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936210 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936213 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936215 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936218 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936221 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936223 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:07:27.087668 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936227 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936230 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936232 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936235 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936238 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936242 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936245 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936247 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936250 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936253 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936256 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936258 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936261 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936263 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936265 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936268 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936271 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936275 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936277 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:07:27.088679 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936280 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936283 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936285 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936288 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936290 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936293 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936296 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936300 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936303 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936306 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936309 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936311 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936314 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936319 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936321 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936324 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936326 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936330 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936333 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:07:27.089692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936335 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936338 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936340 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936343 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936346 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936349 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936351 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936354 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936356 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936359 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936361 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936364 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936368 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936372 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936376 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936380 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936385 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936390 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936393 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:07:27.090692 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936396 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936399 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936401 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936404 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936406 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936408 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936411 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936415 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936418 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936420 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936423 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936427 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936430 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936432 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936435 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936437 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936440 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936443 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936446 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936449 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:07:27.091701 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936451 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:26.936453 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.936458 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.936626 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.939105 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.941010 2582 server.go:1019] "Starting client certificate rotation" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.941174 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.941221 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.944014 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.948079 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.963801 2582 log.go:25] "Validated CRI v1 runtime API" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.966386 2582 log.go:25] "Validated CRI v1 image API" Jul 07 17:07:27.092579 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.967177 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 17:07:27.093006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.969924 2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 dad3bf29-95c9-47f5-8fde-be7528aac0b1:/dev/nvme0n1p4 e62249d5-64e7-48a5-a70b-021dc2ee995d:/dev/nvme0n1p3] Jul 07 17:07:27.093006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.969961 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.976615 2582 manager.go:216] Machine: {Timestamp:2026-07-07 17:07:26.975879278 +0000 UTC m=+0.418598650 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099485 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22956acad63f58a5b9367888f6a33d SystemUUID:ec22956a-cad6-3f58-a5b9-367888f6a33d BootID:f6adfc9e-e126-4a0e-b659-bc778723ae18 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9f:53:f8:00:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9f:53:f8:00:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:a2:12:f1:8d:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.976759 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.976862 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.980427 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.980459 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.980647 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.981321 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.981332 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.981339 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.981353 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.983274 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.987925 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988058 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988091 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988100 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988119 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 17:07:27.100607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988133 2582 kubelet.go:398] "Adding apiserver pod source" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988142 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988448 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.988462 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.991021 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.993548 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.993577 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995739 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995754 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995760 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995767 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995777 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995783 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995790 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995796 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995802 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995809 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995822 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.995827 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.996703 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:26.996711 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.000268 2582 server.go:1268] "Started kubelet" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.000338 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.007058 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.007225 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.007706 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.008066 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.008288 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.008387 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.009994 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.010036 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.011243 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-16.ec2.internal.18c010e962394801 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-16.ec2.internal,UID:ip-10-0-133-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-16.ec2.internal,},FirstTimestamp:2026-07-07 17:07:27.000061953 +0000 UTC m=+0.442781345,LastTimestamp:2026-07-07 17:07:27.000061953 +0000 UTC m=+0.442781345,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-16.ec2.internal,}" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.018293 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.018350 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.019097 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.019098 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.019125 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.019225 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.019234 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.018438 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020585 2582 factory.go:55] Registering systemd factory Jul 07 17:07:27.197241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020600 2582 factory.go:223] Registration of the systemd container factory successfully Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020817 2582 factory.go:153] Registering CRI-O factory Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020831 2582 factory.go:223] Registration of the crio container factory successfully Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020886 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020915 2582 factory.go:103] Registering Raw factory Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.020946 2582 manager.go:1196] Started watching for new ooms in manager Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.020961 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.021082 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.021324 2582 manager.go:349] Starting recovery of all containers Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.052236 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pqpff" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.057768 2582 manager.go:354] Recovery completed Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.058493 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pqpff" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.068227 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071322 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071352 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071364 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071954 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071963 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071980 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.071994 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.072723 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.072747 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.072761 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.074113 2582 policy_none.go:49] "None policy: Start" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.074128 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.074138 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.076106 2582 policy_none.go:47] "Start" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.120965 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.221140 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.229253 2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-16.ec2.internal\" not found" node="ip-10-0-133-16.ec2.internal" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.321259 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.338595 2582 manager.go:337] "Starting Device Plugin manager" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.338687 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.338701 2582 server.go:87] "Starting device plugin registration server" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.338988 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.338999 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.339078 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.339185 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.339195 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.339708 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.339762 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.343778 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.345067 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.345099 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.345122 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.345163 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 17:07:27.469337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.347866 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.439564 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.440655 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.440690 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.440700 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.440727 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-16.ec2.internal" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.445899 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal"] Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.445985 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.449704 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.449734 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.449747 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.452033 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.452129 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.452168 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.452993 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-16.ec2.internal" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.453016 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-16.ec2.internal\": node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.453203 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.453209 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.453239 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.453250 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.453225 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.453304 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.455442 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.455465 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.456160 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.456178 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:07:27.491491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.456192 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:07:27.494471 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.492710 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-16.ec2.internal\" not found" node="ip-10-0-133-16.ec2.internal" Jul 07 17:07:27.497623 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.497477 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-16.ec2.internal\" not found" node="ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.502105 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.522218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b956c94c524f1281d984172ee6c41e32-config\") pod \"kube-apiserver-proxy-ip-10-0-133-16.ec2.internal\" (UID: \"b956c94c524f1281d984172ee6c41e32\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.522242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27ac7136b72eeb4c1bec7700ebb2a216-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal\" (UID: \"27ac7136b72eeb4c1bec7700ebb2a216\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.522259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27ac7136b72eeb4c1bec7700ebb2a216-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal\" (UID: \"27ac7136b72eeb4c1bec7700ebb2a216\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:27.602466 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.622970 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b956c94c524f1281d984172ee6c41e32-config\") pod \"kube-apiserver-proxy-ip-10-0-133-16.ec2.internal\" (UID: \"b956c94c524f1281d984172ee6c41e32\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.622998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27ac7136b72eeb4c1bec7700ebb2a216-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal\" (UID: \"27ac7136b72eeb4c1bec7700ebb2a216\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.623013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27ac7136b72eeb4c1bec7700ebb2a216-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal\" (UID: \"27ac7136b72eeb4c1bec7700ebb2a216\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.623063 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27ac7136b72eeb4c1bec7700ebb2a216-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal\" (UID: \"27ac7136b72eeb4c1bec7700ebb2a216\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.623085 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27ac7136b72eeb4c1bec7700ebb2a216-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal\" (UID: \"27ac7136b72eeb4c1bec7700ebb2a216\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:27.623089 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b956c94c524f1281d984172ee6c41e32-config\") pod \"kube-apiserver-proxy-ip-10-0-133-16.ec2.internal\" (UID: \"b956c94c524f1281d984172ee6c41e32\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.348267 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.285814 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.379312 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.382848 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.386274 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.486971 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.522400 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.522547 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 17:07:29.591881 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.587817 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.864094 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.649698 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 17:02:28 +0000 UTC" deadline="2028-02-22 12:30:03.608669981 +0000 UTC" Jul 07 17:07:29.864094 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.649736 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14275h22m33.958936858s" Jul 07 17:07:29.864094 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.688833 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:29.864094 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.789318 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.889102 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:29.889972 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-16.ec2.internal\" not found" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:29.961738 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.001741 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.009907 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ac7136b72eeb4c1bec7700ebb2a216.slice/crio-2177990356c8ed8ff9543008669420bb5fb6182629c5ad80079deb9989f82feb WatchSource:0}: Error finding container 2177990356c8ed8ff9543008669420bb5fb6182629c5ad80079deb9989f82feb: Status 404 returned error can't find the container with id 2177990356c8ed8ff9543008669420bb5fb6182629c5ad80079deb9989f82feb Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.010355 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb956c94c524f1281d984172ee6c41e32.slice/crio-b5444c90f41835e26efcc44df001f830bcbf218de105957f2274af0f0725cb8d WatchSource:0}: Error finding container b5444c90f41835e26efcc44df001f830bcbf218de105957f2274af0f0725cb8d: Status 404 returned error can't find the container with id b5444c90f41835e26efcc44df001f830bcbf218de105957f2274af0f0725cb8d Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.014437 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.023275 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.024905 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.049395 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 17:07:30.121601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.107849 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.572695 2582 apiserver.go:52] "Watching apiserver" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.584639 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.586086 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-442hr","kube-system/konnectivity-agent-5sj99","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s","openshift-cluster-node-tuning-operator/tuned-9fxjx","openshift-dns/node-resolver-tqtx4","openshift-multus/multus-additional-cni-plugins-bnkgk","openshift-multus/multus-nx9vp","openshift-network-diagnostics/network-check-target-qqgwn","kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal","openshift-image-registry/node-ca-f9gq4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal","openshift-multus/network-metrics-daemon-7n5g9","openshift-network-operator/iptables-alerter-hq6dz","openshift-ovn-kubernetes/ovnkube-node-f446s"] Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.589062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.591060 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.591638 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.591815 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.591875 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.591909 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.592092 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k4snr\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.593317 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.593333 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.593661 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.593961 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xb2rc\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.593963 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.595526 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.595686 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.595834 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.596286 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l2799\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.596503 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.597772 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.597781 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.598341 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bwqpt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.598488 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.600250 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wtm4c\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.600261 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.600478 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.600550 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.602188 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.602645 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.602908 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dm86w\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.602999 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.603220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.637350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.605617 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.605691 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.605766 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.606211 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-46lhr\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.606520 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.606767 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.606816 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.606898 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.606917 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.608178 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.611167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.611546 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.611585 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fmd2k\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612484 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-tmp-dir\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cnibin\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzkfv\" (UniqueName: \"kubernetes.io/projected/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-kube-api-access-pzkfv\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-run-ovn-kubernetes\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612646 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3e77bd34-85f3-4111-a985-a9a12870e2a0-iptables-alerter-script\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612699 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-registration-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612729 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-systemd\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovnkube-script-lib\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612792 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-hosts-file\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.639971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw9bz\" (UniqueName: \"kubernetes.io/projected/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-kube-api-access-pw9bz\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-device-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-node-log\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-run\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhp5t\" (UniqueName: \"kubernetes.io/projected/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-kube-api-access-qhp5t\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.612973 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-sys-fs\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovn-node-metrics-cert\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613033 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-system-cni-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-cni-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-system-cni-dir\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613176 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-slash\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613210 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-os-release\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-var-lib-kubelet\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-netns\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613414 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwwv7\" (UniqueName: \"kubernetes.io/projected/3e77bd34-85f3-4111-a985-a9a12870e2a0-kube-api-access-hwwv7\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvt7n\" (UniqueName: \"kubernetes.io/projected/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-kube-api-access-qvt7n\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-log-socket\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613613 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysconfig\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7dda3f64-b872-4cca-b534-4ca7da05199c-cni-binary-copy\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613695 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-cni-multus\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-multus-certs\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613787 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-kubelet\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-ovn\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pczbn\" (UniqueName: \"kubernetes.io/projected/a33bb79a-1b56-4cb5-9181-ff640db7deca-kube-api-access-pczbn\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysctl-d\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.613992 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-socket-dir-parent\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e77bd34-85f3-4111-a985-a9a12870e2a0-host-slash\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-modprobe-d\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.641978 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614183 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-kubelet\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-env-overrides\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614264 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614284 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614264 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-sys\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614377 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-lib-modules\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614409 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-etc-kubernetes\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-os-release\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-run-netns\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovnkube-config\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614522 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-k8s-cni-cncf-io\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614548 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614551 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-conf-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614603 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-var-lib-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614714 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8fdrb\"" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-cni-bin\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614787 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-kubernetes\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.642953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-tuned\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-socket-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614868 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-hostroot\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614896 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-daemon-config\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krpxc\" (UniqueName: \"kubernetes.io/projected/7dda3f64-b872-4cca-b534-4ca7da05199c-kube-api-access-krpxc\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-cni-netd\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.614977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-tmp\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-cnibin\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615033 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-cni-bin\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615077 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615106 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-systemd-units\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615134 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-etc-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615166 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysctl-conf\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615285 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-systemd\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615336 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-host\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.615452 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:30.644109 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.615522 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:30.645133 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.617889 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:30.645133 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.617954 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:30.645133 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.627692 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 17:07:30.710905 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.671924 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-858n8" Jul 07 17:07:30.710905 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.682742 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-858n8" Jul 07 17:07:30.710905 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.702923 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.715934 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.715975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-modprobe-d\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.715996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-kubelet\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716039 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-env-overrides\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716109 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phlf6\" (UniqueName: \"kubernetes.io/projected/db8bf3da-6be2-460d-b95d-004822f9a091-kube-api-access-phlf6\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716118 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-modprobe-d\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716110 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-kubelet\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716135 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-sys\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-lib-modules\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-etc-kubernetes\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716204 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-os-release\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716239 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-etc-kubernetes\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716248 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-sys\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716282 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-run-netns\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-run-netns\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.716317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovnkube-config\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716345 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-k8s-cni-cncf-io\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716353 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-lib-modules\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-conf-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716393 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-var-lib-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716421 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-os-release\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716463 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-cni-bin\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716472 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-conf-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716485 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-kubernetes\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-tuned\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-socket-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716543 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-k8s-cni-cncf-io\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1387f8c-3973-4bbf-8f83-a0d502302121-agent-certs\") pod \"konnectivity-agent-5sj99\" (UID: \"e1387f8c-3973-4bbf-8f83-a0d502302121\") " pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716598 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-cni-bin\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-hostroot\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.719042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716628 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-daemon-config\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716654 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krpxc\" (UniqueName: \"kubernetes.io/projected/7dda3f64-b872-4cca-b534-4ca7da05199c-kube-api-access-krpxc\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716673 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-env-overrides\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-cni-netd\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-tmp\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-cnibin\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716761 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-kubernetes\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-cni-bin\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716791 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716815 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-systemd-units\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-etc-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716893 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-socket-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716904 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/039a7f67-f570-4358-8404-083823f9c823-dbus\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716913 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-var-lib-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysctl-conf\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716987 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-hostroot\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.720321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717053 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysctl-conf\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717157 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717196 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717212 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-cni-netd\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717243 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717261 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717302 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-systemd-units\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717344 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-etc-openvswitch\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.716872 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovnkube-config\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717404 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-cni-bin\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717484 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-cnibin\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717479 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717506 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-daemon-config\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-systemd\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-host\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-tmp-dir\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717617 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cnibin\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717639 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzkfv\" (UniqueName: \"kubernetes.io/projected/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-kube-api-access-pzkfv\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.721219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-run-ovn-kubernetes\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717696 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3e77bd34-85f3-4111-a985-a9a12870e2a0-iptables-alerter-script\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-registration-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-systemd\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovnkube-script-lib\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-hosts-file\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717853 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw9bz\" (UniqueName: \"kubernetes.io/projected/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-kube-api-access-pw9bz\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717877 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-device-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-node-log\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1387f8c-3973-4bbf-8f83-a0d502302121-konnectivity-ca\") pod \"konnectivity-agent-5sj99\" (UID: \"e1387f8c-3973-4bbf-8f83-a0d502302121\") " pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717956 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-host\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.717953 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-run\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718007 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhp5t\" (UniqueName: \"kubernetes.io/projected/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-kube-api-access-qhp5t\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718014 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-systemd\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718035 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-sys-fs\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718059 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-hosts-file\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovn-node-metrics-cert\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718090 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-run-ovn-kubernetes\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db8bf3da-6be2-460d-b95d-004822f9a091-host\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718088 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cnibin\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718149 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-system-cni-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718161 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-registration-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-cni-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-system-cni-dir\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718205 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-device-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718265 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-slash\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-os-release\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718344 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-var-lib-kubelet\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-netns\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718465 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-run\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.722893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718504 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-slash\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718558 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-os-release\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718653 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718343 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-tmp-dir\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718674 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovnkube-script-lib\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718739 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-var-lib-kubelet\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718746 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-netns\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718749 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718808 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-system-cni-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718821 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-system-cni-dir\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718200 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-systemd\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwwv7\" (UniqueName: \"kubernetes.io/projected/3e77bd34-85f3-4111-a985-a9a12870e2a0-kube-api-access-hwwv7\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718847 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3e77bd34-85f3-4111-a985-a9a12870e2a0-iptables-alerter-script\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvt7n\" (UniqueName: \"kubernetes.io/projected/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-kube-api-access-qvt7n\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718867 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-cni-dir\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718896 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-sys-fs\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718897 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-node-log\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.723801 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.718976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-log-socket\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719009 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/039a7f67-f570-4358-8404-083823f9c823-kubelet-config\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719038 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-log-socket\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719042 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/db8bf3da-6be2-460d-b95d-004822f9a091-serviceca\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysconfig\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719125 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7dda3f64-b872-4cca-b534-4ca7da05199c-cni-binary-copy\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719152 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-cni-multus\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719188 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysconfig\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719227 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-multus-certs\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719245 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-var-lib-cni-multus\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719294 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-kubelet\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719343 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-host-run-multus-certs\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719342 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-host-kubelet\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-ovn\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pczbn\" (UniqueName: \"kubernetes.io/projected/a33bb79a-1b56-4cb5-9181-ff640db7deca-kube-api-access-pczbn\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysctl-d\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.724724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-socket-dir-parent\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719513 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a33bb79a-1b56-4cb5-9181-ff640db7deca-run-ovn\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719519 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e77bd34-85f3-4111-a985-a9a12870e2a0-host-slash\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719558 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e77bd34-85f3-4111-a985-a9a12870e2a0-host-slash\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719628 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7dda3f64-b872-4cca-b534-4ca7da05199c-multus-socket-dir-parent\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719629 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppkh\" (UniqueName: \"kubernetes.io/projected/158c3d23-9652-4f2b-acee-d535a4f662f6-kube-api-access-wppkh\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719667 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-sysctl-d\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719563 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7dda3f64-b872-4cca-b534-4ca7da05199c-cni-binary-copy\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719673 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.719863 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.721496 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-etc-tuned\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.721529 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-tmp\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.725882 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.722160 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a33bb79a-1b56-4cb5-9181-ff640db7deca-ovn-node-metrics-cert\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.736887 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzkfv\" (UniqueName: \"kubernetes.io/projected/06ec47f1-f20a-4b8f-b7a7-81751a15cdc3-kube-api-access-pzkfv\") pod \"multus-additional-cni-plugins-bnkgk\" (UID: \"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3\") " pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.741409 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvt7n\" (UniqueName: \"kubernetes.io/projected/a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89-kube-api-access-qvt7n\") pod \"aws-ebs-csi-driver-node-t6k7s\" (UID: \"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.741355 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwwv7\" (UniqueName: \"kubernetes.io/projected/3e77bd34-85f3-4111-a985-a9a12870e2a0-kube-api-access-hwwv7\") pod \"iptables-alerter-hq6dz\" (UID: \"3e77bd34-85f3-4111-a985-a9a12870e2a0\") " pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.742733 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw9bz\" (UniqueName: \"kubernetes.io/projected/3ff53b6a-4006-4d81-88c4-6cae9a6590ef-kube-api-access-pw9bz\") pod \"node-resolver-tqtx4\" (UID: \"3ff53b6a-4006-4d81-88c4-6cae9a6590ef\") " pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.743353 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krpxc\" (UniqueName: \"kubernetes.io/projected/7dda3f64-b872-4cca-b534-4ca7da05199c-kube-api-access-krpxc\") pod \"multus-nx9vp\" (UID: \"7dda3f64-b872-4cca-b534-4ca7da05199c\") " pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.744013 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pczbn\" (UniqueName: \"kubernetes.io/projected/a33bb79a-1b56-4cb5-9181-ff640db7deca-kube-api-access-pczbn\") pod \"ovnkube-node-f446s\" (UID: \"a33bb79a-1b56-4cb5-9181-ff640db7deca\") " pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.744602 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhp5t\" (UniqueName: \"kubernetes.io/projected/e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f-kube-api-access-qhp5t\") pod \"tuned-9fxjx\" (UID: \"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f\") " pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.763125 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.820771 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppkh\" (UniqueName: \"kubernetes.io/projected/158c3d23-9652-4f2b-acee-d535a4f662f6-kube-api-access-wppkh\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.820828 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.820874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phlf6\" (UniqueName: \"kubernetes.io/projected/db8bf3da-6be2-460d-b95d-004822f9a091-kube-api-access-phlf6\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.820909 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1387f8c-3973-4bbf-8f83-a0d502302121-agent-certs\") pod \"konnectivity-agent-5sj99\" (UID: \"e1387f8c-3973-4bbf-8f83-a0d502302121\") " pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.820961 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/039a7f67-f570-4358-8404-083823f9c823-dbus\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.820996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1387f8c-3973-4bbf-8f83-a0d502302121-konnectivity-ca\") pod \"konnectivity-agent-5sj99\" (UID: \"e1387f8c-3973-4bbf-8f83-a0d502302121\") " pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db8bf3da-6be2-460d-b95d-004822f9a091-host\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821064 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:30.827777 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821121 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/039a7f67-f570-4358-8404-083823f9c823-kubelet-config\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821154 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/db8bf3da-6be2-460d-b95d-004822f9a091-serviceca\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821498 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/039a7f67-f570-4358-8404-083823f9c823-dbus\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.821693 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/db8bf3da-6be2-460d-b95d-004822f9a091-serviceca\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.821883 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.821976 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:31.321940078 +0000 UTC m=+3.182020143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.822196 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1387f8c-3973-4bbf-8f83-a0d502302121-konnectivity-ca\") pod \"konnectivity-agent-5sj99\" (UID: \"e1387f8c-3973-4bbf-8f83-a0d502302121\") " pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.822266 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db8bf3da-6be2-460d-b95d-004822f9a091-host\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.822612 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.822675 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:31.322644172 +0000 UTC m=+3.182724222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.822732 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/039a7f67-f570-4358-8404-083823f9c823-kubelet-config\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:30.829027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.826277 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1387f8c-3973-4bbf-8f83-a0d502302121-agent-certs\") pod \"konnectivity-agent-5sj99\" (UID: \"e1387f8c-3973-4bbf-8f83-a0d502302121\") " pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.832279 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.832140 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:07:30.832279 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.832164 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:07:30.832279 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.832179 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:30.832279 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:30.832232 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:31.332217362 +0000 UTC m=+3.192297424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:30.833437 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.833020 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppkh\" (UniqueName: \"kubernetes.io/projected/158c3d23-9652-4f2b-acee-d535a4f662f6-kube-api-access-wppkh\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:30.898537 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.835386 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phlf6\" (UniqueName: \"kubernetes.io/projected/db8bf3da-6be2-460d-b95d-004822f9a091-kube-api-access-phlf6\") pod \"node-ca-f9gq4\" (UID: \"db8bf3da-6be2-460d-b95d-004822f9a091\") " pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.901261 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.899587 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nx9vp" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.908325 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hq6dz" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.916238 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dda3f64_b872_4cca_b534_4ca7da05199c.slice/crio-e00b17ce56629e1a032f5793c8b1758f4f878dbbc2f5fdd6a39afc581600ca8d WatchSource:0}: Error finding container e00b17ce56629e1a032f5793c8b1758f4f878dbbc2f5fdd6a39afc581600ca8d: Status 404 returned error can't find the container with id e00b17ce56629e1a032f5793c8b1758f4f878dbbc2f5fdd6a39afc581600ca8d Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.916929 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.923044 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e77bd34_85f3_4111_a985_a9a12870e2a0.slice/crio-5bd317a1baf8c6c56b8c3435447f4edda13d1c99994923f28a78a6858a0d9c29 WatchSource:0}: Error finding container 5bd317a1baf8c6c56b8c3435447f4edda13d1c99994923f28a78a6858a0d9c29: Status 404 returned error can't find the container with id 5bd317a1baf8c6c56b8c3435447f4edda13d1c99994923f28a78a6858a0d9c29 Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.925779 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.930840 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda126fa1f_6a7c_4bdd_9d94_b095ab7cdc89.slice/crio-bb5ab1b05323538f7ee9a16b2f482df7b2343b28cd8b9dcf3b038819f6170cc0 WatchSource:0}: Error finding container bb5ab1b05323538f7ee9a16b2f482df7b2343b28cd8b9dcf3b038819f6170cc0: Status 404 returned error can't find the container with id bb5ab1b05323538f7ee9a16b2f482df7b2343b28cd8b9dcf3b038819f6170cc0 Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.934297 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tqtx4" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.935159 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hq6dz" event={"ID":"3e77bd34-85f3-4111-a985-a9a12870e2a0","Type":"ContainerStarted","Data":"5bd317a1baf8c6c56b8c3435447f4edda13d1c99994923f28a78a6858a0d9c29"} Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.936484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nx9vp" event={"ID":"7dda3f64-b872-4cca-b534-4ca7da05199c","Type":"ContainerStarted","Data":"e00b17ce56629e1a032f5793c8b1758f4f878dbbc2f5fdd6a39afc581600ca8d"} Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.937410 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7496dfb_0b4c_4aa3_ac8b_2e736ef7b69f.slice/crio-d23ad56c12ae28373fa9313fbdba8a503b0bf5761c64aaa889bcf8e57cc037f0 WatchSource:0}: Error finding container d23ad56c12ae28373fa9313fbdba8a503b0bf5761c64aaa889bcf8e57cc037f0: Status 404 returned error can't find the container with id d23ad56c12ae28373fa9313fbdba8a503b0bf5761c64aaa889bcf8e57cc037f0 Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.938353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" event={"ID":"27ac7136b72eeb4c1bec7700ebb2a216","Type":"ContainerStarted","Data":"2177990356c8ed8ff9543008669420bb5fb6182629c5ad80079deb9989f82feb"} Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.940887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" event={"ID":"b956c94c524f1281d984172ee6c41e32","Type":"ContainerStarted","Data":"b5444c90f41835e26efcc44df001f830bcbf218de105957f2274af0f0725cb8d"} Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.941305 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.946210 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff53b6a_4006_4d81_88c4_6cae9a6590ef.slice/crio-64a3fe10ab1d0733cd47f575fe584d253c6320c5e63f6d08e5f0106442f89603 WatchSource:0}: Error finding container 64a3fe10ab1d0733cd47f575fe584d253c6320c5e63f6d08e5f0106442f89603: Status 404 returned error can't find the container with id 64a3fe10ab1d0733cd47f575fe584d253c6320c5e63f6d08e5f0106442f89603 Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.950307 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.952992 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ec47f1_f20a_4b8f_b7a7_81751a15cdc3.slice/crio-11252a295c225dac0296f8c6be1266034920c4504240061d88b717cd6a303536 WatchSource:0}: Error finding container 11252a295c225dac0296f8c6be1266034920c4504240061d88b717cd6a303536: Status 404 returned error can't find the container with id 11252a295c225dac0296f8c6be1266034920c4504240061d88b717cd6a303536 Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.956158 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.960527 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda33bb79a_1b56_4cb5_9181_ff640db7deca.slice/crio-cb8cd6fe07dc8fbbc18c50da8e10fb73e98882ea87d098124dfd47107c264a05 WatchSource:0}: Error finding container cb8cd6fe07dc8fbbc18c50da8e10fb73e98882ea87d098124dfd47107c264a05: Status 404 returned error can't find the container with id cb8cd6fe07dc8fbbc18c50da8e10fb73e98882ea87d098124dfd47107c264a05 Jul 07 17:07:30.971822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:30.963848 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f9gq4" Jul 07 17:07:30.976522 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.976118 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1387f8c_3973_4bbf_8f83_a0d502302121.slice/crio-090872580e0c02c840cc9f03691e9829b0b4f9f57358b7a964512c2d27192e04 WatchSource:0}: Error finding container 090872580e0c02c840cc9f03691e9829b0b4f9f57358b7a964512c2d27192e04: Status 404 returned error can't find the container with id 090872580e0c02c840cc9f03691e9829b0b4f9f57358b7a964512c2d27192e04 Jul 07 17:07:31.067811 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:07:30.981054 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb8bf3da_6be2_460d_b95d_004822f9a091.slice/crio-2ed91f2b094e3a71832b99393d2594af5aa3fe5c3a6d105190f8274269637331 WatchSource:0}: Error finding container 2ed91f2b094e3a71832b99393d2594af5aa3fe5c3a6d105190f8274269637331: Status 404 returned error can't find the container with id 2ed91f2b094e3a71832b99393d2594af5aa3fe5c3a6d105190f8274269637331 Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.324482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.324535 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.324663 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.324730 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:32.324711151 +0000 UTC m=+4.184791208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.324669 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.324796 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:32.324783368 +0000 UTC m=+4.184863418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.408078 2582 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517" Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.408246 2582 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-nx9vp_openshift-multus(7dda3f64-b872-4cca-b534-4ca7da05199c): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 07 17:07:31.410349 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.408298 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-multus/multus-nx9vp" podUID="7dda3f64-b872-4cca-b534-4ca7da05199c" Jul 07 17:07:31.425175 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.425148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:31.425330 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.425312 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:07:31.425374 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.425337 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:07:31.425374 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.425349 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:31.425441 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.425398 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:32.425383514 +0000 UTC m=+4.285463564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:31.685862 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.685818 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:02:30 +0000 UTC" deadline="2028-04-08 19:25:18.189337074 +0000 UTC" Jul 07 17:07:31.685862 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.685861 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15386h17m46.50348117s" Jul 07 17:07:31.929312 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.929278 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:31.929499 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.929411 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:31.929846 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.929825 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:31.929959 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.929936 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:31.947001 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.946930 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f9gq4" event={"ID":"db8bf3da-6be2-460d-b95d-004822f9a091","Type":"ContainerStarted","Data":"2ed91f2b094e3a71832b99393d2594af5aa3fe5c3a6d105190f8274269637331"} Jul 07 17:07:31.951445 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.951415 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"cb8cd6fe07dc8fbbc18c50da8e10fb73e98882ea87d098124dfd47107c264a05"} Jul 07 17:07:31.953968 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.953943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerStarted","Data":"11252a295c225dac0296f8c6be1266034920c4504240061d88b717cd6a303536"} Jul 07 17:07:31.955612 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.955564 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" event={"ID":"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f","Type":"ContainerStarted","Data":"d23ad56c12ae28373fa9313fbdba8a503b0bf5761c64aaa889bcf8e57cc037f0"} Jul 07 17:07:31.959414 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.959371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" event={"ID":"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89","Type":"ContainerStarted","Data":"bb5ab1b05323538f7ee9a16b2f482df7b2343b28cd8b9dcf3b038819f6170cc0"} Jul 07 17:07:31.964639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.964612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5sj99" event={"ID":"e1387f8c-3973-4bbf-8f83-a0d502302121","Type":"ContainerStarted","Data":"090872580e0c02c840cc9f03691e9829b0b4f9f57358b7a964512c2d27192e04"} Jul 07 17:07:31.966830 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:31.966806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tqtx4" event={"ID":"3ff53b6a-4006-4d81-88c4-6cae9a6590ef","Type":"ContainerStarted","Data":"64a3fe10ab1d0733cd47f575fe584d253c6320c5e63f6d08e5f0106442f89603"} Jul 07 17:07:31.969541 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:31.969318 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-multus/multus-nx9vp" podUID="7dda3f64-b872-4cca-b534-4ca7da05199c" Jul 07 17:07:32.333311 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:32.333225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:32.333311 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:32.333285 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:32.333526 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.333419 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:32.333526 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.333481 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:34.333463115 +0000 UTC m=+6.193543170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:32.333749 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.333734 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:32.333812 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.333794 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:34.333779153 +0000 UTC m=+6.193859208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:32.434224 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:32.434146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:32.434382 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.434358 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:07:32.434452 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.434390 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:07:32.434452 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.434404 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:32.434566 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.434465 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:34.434445995 +0000 UTC m=+6.294526048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:32.686222 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:32.686155 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:02:30 +0000 UTC" deadline="2028-01-05 05:53:44.030053141 +0000 UTC" Jul 07 17:07:32.686222 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:32.686211 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13116h46m11.343846119s" Jul 07 17:07:32.931271 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:32.930788 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:32.931271 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:32.930929 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:33.928599 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:33.928293 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:33.928599 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:33.928418 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:33.928599 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:33.928441 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:33.928599 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:33.928538 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:34.348731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:34.348642 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:34.348731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:34.348691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:34.348946 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.348812 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:34.348946 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.348887 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:38.348866415 +0000 UTC m=+10.208946481 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:34.348946 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.348812 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:34.349148 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.348965 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:38.348949194 +0000 UTC m=+10.209029265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:34.449464 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:34.449423 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:34.449645 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.449603 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:07:34.449645 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.449622 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:07:34.449645 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.449635 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:34.449766 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.449695 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:38.449678537 +0000 UTC m=+10.309758589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:34.928525 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:34.928479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:34.928722 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:34.928657 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:35.928773 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:35.928731 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:35.929232 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:35.928731 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:35.929232 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:35.928876 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:35.929232 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:35.928921 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:36.928529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:36.928494 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:36.928718 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:36.928673 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:37.928650 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:37.928608 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:37.928650 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:37.928648 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:37.929163 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:37.928738 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:37.929163 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:37.928855 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:38.378972 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:38.378930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:38.379159 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:38.379026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:38.379218 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.379168 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:38.379218 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.379199 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:38.379323 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.379250 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:46.379229132 +0000 UTC m=+18.239309186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:38.379323 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.379268 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:46.379259172 +0000 UTC m=+18.239339228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:38.479811 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:38.479771 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:38.480002 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.479921 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:07:38.480002 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.479939 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:07:38.480002 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.479949 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:38.480146 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.480005 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:46.479991397 +0000 UTC m=+18.340071450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:38.928714 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:38.928683 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:38.929131 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:38.928813 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:39.928680 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:39.928651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:39.928857 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:39.928748 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:39.928857 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:39.928816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:39.929206 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:39.928956 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:40.929201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.929013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:40.929601 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:40.929328 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:40.986121 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.986085 2582 generic.go:356] "Generic (PLEG): container finished" podID="27ac7136b72eeb4c1bec7700ebb2a216" containerID="55fecf715fa74788e8aecf74d990717a95e4926e9a806ead2ba12aa37d4113a1" exitCode=0 Jul 07 17:07:40.986266 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.986171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" event={"ID":"27ac7136b72eeb4c1bec7700ebb2a216","Type":"ContainerDied","Data":"55fecf715fa74788e8aecf74d990717a95e4926e9a806ead2ba12aa37d4113a1"} Jul 07 17:07:40.987991 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.987965 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" event={"ID":"b956c94c524f1281d984172ee6c41e32","Type":"ContainerStarted","Data":"91524ec6fa7e8bc1f48c2e4c8162979f97aa74d1bf9f7958e2f640e590eb7851"} Jul 07 17:07:40.989813 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.989790 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f9gq4" event={"ID":"db8bf3da-6be2-460d-b95d-004822f9a091","Type":"ContainerStarted","Data":"5f04ef6f2e37d26c8536e7b83ba8a065f51780014ade7111b4df1c33081ce478"} Jul 07 17:07:40.991632 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.991603 2582 generic.go:356] "Generic (PLEG): container finished" podID="06ec47f1-f20a-4b8f-b7a7-81751a15cdc3" containerID="a6bade474007ca62833e13fea0e32272abd7f546a9cbd6fe26b18df218bfa4e4" exitCode=0 Jul 07 17:07:40.991718 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.991690 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerDied","Data":"a6bade474007ca62833e13fea0e32272abd7f546a9cbd6fe26b18df218bfa4e4"} Jul 07 17:07:40.993545 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.993521 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" event={"ID":"e7496dfb-0b4c-4aa3-ac8b-2e736ef7b69f","Type":"ContainerStarted","Data":"3bdc06e26bd49cb6be86aeba3ced4767efb023267e4813633adbca771c475396"} Jul 07 17:07:40.995445 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.995418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" event={"ID":"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89","Type":"ContainerStarted","Data":"66cc99abf67562b43d6afb737727ac117ad1c7b46b5ea7b2520509305bfab144"} Jul 07 17:07:40.997765 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:40.997356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5sj99" event={"ID":"e1387f8c-3973-4bbf-8f83-a0d502302121","Type":"ContainerStarted","Data":"af5eda6b73352f633bf3302f123827bbf121e259a3137bfc86e0eb446d599532"} Jul 07 17:07:41.001192 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.001167 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tqtx4" event={"ID":"3ff53b6a-4006-4d81-88c4-6cae9a6590ef","Type":"ContainerStarted","Data":"261286400eab9cd74ae9f71674cde2972789d739e5cd417a44ea2555054d8c4a"} Jul 07 17:07:41.018753 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.018701 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-16.ec2.internal" podStartSLOduration=11.01868454 podStartE2EDuration="11.01868454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:41.018527541 +0000 UTC m=+12.878607614" watchObservedRunningTime="2026-07-07 17:07:41.01868454 +0000 UTC m=+12.878764615" Jul 07 17:07:41.075157 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.075111 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5sj99" podStartSLOduration=2.845970752 podStartE2EDuration="12.075097754s" totalImagesPullingTime="9.229127002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:41.073148358 +0000 UTC m=+12.933228426" watchObservedRunningTime="2026-07-07 17:07:41.075097754 +0000 UTC m=+12.935177856" Jul 07 17:07:41.075297 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.075278 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tqtx4" podStartSLOduration=2.827748293 podStartE2EDuration="12.075272837s" totalImagesPullingTime="9.247524544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:41.052342521 +0000 UTC m=+12.912422596" watchObservedRunningTime="2026-07-07 17:07:41.075272837 +0000 UTC m=+12.935352908" Jul 07 17:07:41.101271 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.099550 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9fxjx" podStartSLOduration=2.825765841 podStartE2EDuration="12.099531889s" totalImagesPullingTime="9.273766048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:41.098284576 +0000 UTC m=+12.958364648" watchObservedRunningTime="2026-07-07 17:07:41.099531889 +0000 UTC m=+12.959611972" Jul 07 17:07:41.180846 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.180603 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f9gq4" podStartSLOduration=2.971292844 podStartE2EDuration="12.180588265s" totalImagesPullingTime="9.209295421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:41.180445814 +0000 UTC m=+13.040525886" watchObservedRunningTime="2026-07-07 17:07:41.180588265 +0000 UTC m=+13.040668328" Jul 07 17:07:41.422006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.421965 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:41.422703 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.422679 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:41.928076 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.928042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:41.928261 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:41.928177 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:41.928618 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:41.928593 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:41.928741 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:41.928699 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:42.005051 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.004964 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hq6dz" event={"ID":"3e77bd34-85f3-4111-a985-a9a12870e2a0","Type":"ContainerStarted","Data":"74eef936f5540e4c39d302057c7f4409d4063cef4b9d938065174b9a05863acb"} Jul 07 17:07:42.006046 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.006023 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:42.006685 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.006663 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5sj99" Jul 07 17:07:42.053268 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.053221 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hq6dz" podStartSLOduration=3.781684734 podStartE2EDuration="13.053201578s" totalImagesPullingTime="9.271516844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:42.025782012 +0000 UTC m=+13.885862084" watchObservedRunningTime="2026-07-07 17:07:42.053201578 +0000 UTC m=+13.913281651" Jul 07 17:07:42.451041 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.451021 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 17:07:42.927899 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.927783 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T17:07:42.451036969Z","UUID":"727c7fe4-da34-447a-aa24-330ec3d9fbd1","Handler":null,"Name":"","Endpoint":""} Jul 07 17:07:42.928287 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.928265 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:42.928431 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:42.928407 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:42.929871 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.929844 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 17:07:42.929992 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:42.929895 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 17:07:43.010372 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:43.010334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" event={"ID":"27ac7136b72eeb4c1bec7700ebb2a216","Type":"ContainerStarted","Data":"e338a3b8b539700ed1ee0c706d3a0e193594cb3c1c5bf48fa38a772c47f376c9"} Jul 07 17:07:43.013537 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:43.013504 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" event={"ID":"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89","Type":"ContainerStarted","Data":"62c6cd2df81d26ba8f48f2b23f59b2eabbeba4b51e0881979ca87248e682230b"} Jul 07 17:07:43.037016 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:43.036972 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-16.ec2.internal" podStartSLOduration=13.036956661 podStartE2EDuration="13.036956661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:43.036765742 +0000 UTC m=+14.896845815" watchObservedRunningTime="2026-07-07 17:07:43.036956661 +0000 UTC m=+14.897036732" Jul 07 17:07:43.928176 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:43.928149 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:43.928338 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:43.928254 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:43.928338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:43.928305 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:43.928415 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:43.928397 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:44.928205 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:44.928171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:44.928817 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:44.928310 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:45.020510 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:45.020247 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" event={"ID":"a126fa1f-6a7c-4bdd-9d94-b095ab7cdc89","Type":"ContainerStarted","Data":"a0a589ef4ab3f7d413be5ce61dcb57dfe24d4fd42a617a525c988a38505539ca"} Jul 07 17:07:45.046347 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:45.046296 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6k7s" podStartSLOduration=3.454784003 podStartE2EDuration="16.046275864s" totalImagesPullingTime="12.591491861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:45.04594261 +0000 UTC m=+16.906022683" watchObservedRunningTime="2026-07-07 17:07:45.046275864 +0000 UTC m=+16.906355937" Jul 07 17:07:45.928481 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:45.928453 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:45.928987 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:45.928461 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:45.928987 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:45.928966 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:45.929510 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:45.929474 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:46.443889 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:46.443861 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:46.444051 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:46.443899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:46.444051 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.444005 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:46.444177 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.444067 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:02.444048319 +0000 UTC m=+34.304128389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:07:46.444177 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.444007 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:46.444177 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.444134 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:02.444120064 +0000 UTC m=+34.304200117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:07:46.544370 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:46.544337 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:46.544548 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.544504 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:07:46.544548 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.544525 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:07:46.544548 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.544539 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:46.544730 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.544615 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:02.544595168 +0000 UTC m=+34.404675232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:07:46.928334 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:46.928296 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:46.928515 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:46.928445 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:47.928821 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:47.928779 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:47.929335 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:47.928916 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:47.929335 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:47.928970 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:47.929335 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:47.929081 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:48.929089 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:48.929063 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:48.929527 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:48.929196 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:49.928407 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:49.928373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:49.928545 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:49.928373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:49.928545 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:49.928480 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:49.928620 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:49.928560 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:50.928308 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:50.928072 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:50.928749 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:50.928439 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:51.037653 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.037612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"88392abac6cb31f30857be338f23933d0d8c6224210c4c2052d8e07f0542ab53"} Jul 07 17:07:51.037803 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.037722 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"c38b2a2daebc91f39e75e8fe348cae206459607ddcc0a21425ec93f48029d499"} Jul 07 17:07:51.037803 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.037757 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"979de2c75a17874a3473e6d267ee6d43809448fec68d6fa52dd19f2eae814f99"} Jul 07 17:07:51.037904 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.037805 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"9d0de0f8a031c21fe2dbf5aba006c87e7dd54855e3de5e97e1eccf9d19cf74f7"} Jul 07 17:07:51.037904 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.037836 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"f86576ec30be04c5dbbf6d40d1be5f74a3bda05a3fb684c8cee89e7645fb3628"} Jul 07 17:07:51.037904 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.037869 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"11b0eaeb94de3800566f9b019eb3d8ebec985adbbc316d385eeef5c047c8653c"} Jul 07 17:07:51.039272 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.039251 2582 generic.go:356] "Generic (PLEG): container finished" podID="06ec47f1-f20a-4b8f-b7a7-81751a15cdc3" containerID="8c4e51f99a5fa06f88442eec732e017f82adf12e6a0644a398834e8105cbcb44" exitCode=0 Jul 07 17:07:51.039312 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.039291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerDied","Data":"8c4e51f99a5fa06f88442eec732e017f82adf12e6a0644a398834e8105cbcb44"} Jul 07 17:07:51.928133 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.928098 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:51.928329 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:51.928098 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:51.928329 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:51.928237 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:51.928329 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:51.928267 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:52.928510 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:52.928475 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:52.928917 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:52.928630 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:53.047071 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:53.047038 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"3c6ff37a4f6eb7077ebd284b24fb4591d64432117798aa8e70522e99debff97e"} Jul 07 17:07:53.048907 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:53.048882 2582 generic.go:356] "Generic (PLEG): container finished" podID="06ec47f1-f20a-4b8f-b7a7-81751a15cdc3" containerID="0a0bab80c783d2009e518475017cadd3781f3e3b7fb6610d4ebecc5a1cc1222b" exitCode=0 Jul 07 17:07:53.049028 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:53.048933 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerDied","Data":"0a0bab80c783d2009e518475017cadd3781f3e3b7fb6610d4ebecc5a1cc1222b"} Jul 07 17:07:53.928376 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:53.928349 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:53.928474 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:53.928392 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:53.928510 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:53.928473 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:53.928608 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:53.928586 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:54.052858 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:54.052768 2582 generic.go:356] "Generic (PLEG): container finished" podID="06ec47f1-f20a-4b8f-b7a7-81751a15cdc3" containerID="722d628fc73843a1ed606f81596e4289332a0535cdb2e4c69cf9fac1e41656e8" exitCode=0 Jul 07 17:07:54.052858 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:54.052824 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerDied","Data":"722d628fc73843a1ed606f81596e4289332a0535cdb2e4c69cf9fac1e41656e8"} Jul 07 17:07:54.928184 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:54.927988 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:54.928338 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:54.928270 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:55.060300 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:55.060238 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" event={"ID":"a33bb79a-1b56-4cb5-9181-ff640db7deca","Type":"ContainerStarted","Data":"547ac58af2724845e07f98eab6a8d95b53235d1b69b3fd5235eec929cae3fe73"} Jul 07 17:07:55.061001 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:55.060642 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:55.080283 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:55.080255 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:55.099236 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:55.099182 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" podStartSLOduration=7.055035436 podStartE2EDuration="26.099161893s" totalImagesPullingTime="19.044126457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:07:55.098396951 +0000 UTC m=+26.958477024" watchObservedRunningTime="2026-07-07 17:07:55.099161893 +0000 UTC m=+26.959241965" Jul 07 17:07:55.928820 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:55.928782 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:55.928968 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:55.928888 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:55.928968 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:55.928943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:55.929148 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:55.929125 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:56.062953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.062920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:56.063369 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.063004 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:56.079692 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.079653 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:07:56.577466 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.577428 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7n5g9"] Jul 07 17:07:56.577711 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.577602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:56.577785 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:56.577712 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:56.580843 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.580674 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-442hr"] Jul 07 17:07:56.580843 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.580766 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:56.581139 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:56.581114 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:56.584702 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.584679 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qqgwn"] Jul 07 17:07:56.584810 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:56.584769 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:56.584881 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:56.584860 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:57.928254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:57.928223 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:57.928743 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:57.928331 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:07:58.929061 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:58.928946 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:07:58.929509 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:58.929051 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:07:58.929509 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:58.929073 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:07:58.929509 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:58.929190 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:07:59.928038 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:07:59.927997 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:07:59.928204 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:07:59.928114 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:00.929143 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:00.928874 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:00.929636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:00.928942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:00.929636 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:00.929273 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:00.929636 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:00.929409 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:01.080491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:01.080373 2582 generic.go:356] "Generic (PLEG): container finished" podID="06ec47f1-f20a-4b8f-b7a7-81751a15cdc3" containerID="25f3e57b777dccc0de2b5c86cbdeabc37313d8fb2b38b87fae716dc76a1b3d86" exitCode=0 Jul 07 17:08:01.080491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:01.080413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerDied","Data":"25f3e57b777dccc0de2b5c86cbdeabc37313d8fb2b38b87fae716dc76a1b3d86"} Jul 07 17:08:01.928623 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:01.928591 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:01.928868 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:01.928718 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:02.085344 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.085309 2582 generic.go:356] "Generic (PLEG): container finished" podID="06ec47f1-f20a-4b8f-b7a7-81751a15cdc3" containerID="8fc984da2d47b2c482f42d8326569fa46e4ecb56e3c5849212a2b52c80dcf076" exitCode=0 Jul 07 17:08:02.085828 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.085358 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerDied","Data":"8fc984da2d47b2c482f42d8326569fa46e4ecb56e3c5849212a2b52c80dcf076"} Jul 07 17:08:02.466682 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.466641 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:02.466876 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.466738 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:02.466876 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.466777 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:08:02.466876 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.466851 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret podName:039a7f67-f570-4358-8404-083823f9c823 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:34.466835351 +0000 UTC m=+66.326915424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret") pod "global-pull-secret-syncer-442hr" (UID: "039a7f67-f570-4358-8404-083823f9c823") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:08:02.466876 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.466863 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:08:02.467053 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.466917 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:34.466900342 +0000 UTC m=+66.326980392 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:08:02.567195 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.567155 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:02.567358 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.567314 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:08:02.567358 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.567338 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:08:02.567358 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.567347 2582 projected.go:196] Error preparing data for projected volume kube-api-access-b62jc for pod openshift-network-diagnostics/network-check-target-qqgwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:08:02.567460 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.567400 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc podName:c3af20bb-2a20-4a61-87e3-fd19895e3a70 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:34.567388443 +0000 UTC m=+66.427468492 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b62jc" (UniqueName: "kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc") pod "network-check-target-qqgwn" (UID: "c3af20bb-2a20-4a61-87e3-fd19895e3a70") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:08:02.928769 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.928732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:02.928967 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:02.928732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:02.928967 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.928871 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:02.928967 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:02.928904 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:03.928826 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:03.928796 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:03.929251 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:03.928927 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:04.928772 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:04.928738 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:04.928959 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:04.928869 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:04.929298 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:04.928959 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:04.929298 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:04.929064 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:05.928742 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:05.928714 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:05.928982 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:05.928833 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:06.928584 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:06.928534 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:06.928728 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:06.928666 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:06.928728 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:06.928702 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:06.928812 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:06.928770 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:07.928082 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:07.928047 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:07.928684 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:07.928174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:08.931693 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:08.931663 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:08.932159 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:08.931704 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:08.932159 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:08.931778 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:08.932159 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:08.931916 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:09.928655 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:09.928617 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:09.928840 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:09.928749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:10.932858 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:10.932828 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:10.932858 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:10.932861 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:10.933426 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:10.932828 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:10.933426 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:10.932947 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:10.933426 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:10.933060 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:10.933426 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:10.933171 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:12.928118 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:12.928078 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:12.928759 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:12.928199 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:12.928759 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:12.928211 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqgwn" podUID="c3af20bb-2a20-4a61-87e3-fd19895e3a70" Jul 07 17:08:12.928759 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:12.928317 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-442hr" podUID="039a7f67-f570-4358-8404-083823f9c823" Jul 07 17:08:12.928759 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:12.928398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:12.928759 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:12.928674 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7n5g9" podUID="158c3d23-9652-4f2b-acee-d535a4f662f6" Jul 07 17:08:13.113989 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:13.113954 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" event={"ID":"06ec47f1-f20a-4b8f-b7a7-81751a15cdc3","Type":"ContainerStarted","Data":"6596c3d450d9899e65a0f0bd96ceefd4ad2e7417393d0dcc814f4c1cd9f5e689"} Jul 07 17:08:13.115410 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:13.115386 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nx9vp" event={"ID":"7dda3f64-b872-4cca-b534-4ca7da05199c","Type":"ContainerStarted","Data":"7a679b98e09b9270839b8929c58820f8e05a1bc827edc0f7d61f7cbb13e9556f"} Jul 07 17:08:13.154078 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:13.154030 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bnkgk" podStartSLOduration=7.473028604 podStartE2EDuration="44.154016116s" totalImagesPullingTime="36.680987512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=6 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:13.153468298 +0000 UTC m=+45.013548368" watchObservedRunningTime="2026-07-07 17:08:13.154016116 +0000 UTC m=+45.014096187" Jul 07 17:08:13.966274 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:13.966240 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-16.ec2.internal" event="NodeReady" Jul 07 17:08:13.966639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:13.966366 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 17:08:14.020616 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.020531 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nx9vp" podStartSLOduration=3.908614237 podStartE2EDuration="45.020513953s" totalImagesPullingTime="41.111899716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-07 17:08:13.186089075 +0000 UTC m=+45.046169148" watchObservedRunningTime="2026-07-07 17:08:14.020513953 +0000 UTC m=+45.880594025" Jul 07 17:08:14.021059 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.021033 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67447d8bcd-fqdlm"] Jul 07 17:08:14.036522 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.036501 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.050567 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.050543 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4jnln"] Jul 07 17:08:14.056526 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.056508 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wbvmj"] Jul 07 17:08:14.056660 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.056644 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.060137 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.060115 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 17:08:14.060701 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.060686 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 17:08:14.060756 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.060715 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n59wn\"" Jul 07 17:08:14.061337 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.061319 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 17:08:14.062602 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.062542 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67447d8bcd-fqdlm"] Jul 07 17:08:14.062664 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.062655 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.065923 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.065901 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 17:08:14.066106 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.066092 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xf5p6\"" Jul 07 17:08:14.066153 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.066095 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 17:08:14.066297 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.066278 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r2xr\"" Jul 07 17:08:14.066400 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.066326 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 17:08:14.066400 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.066326 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 17:08:14.070478 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.070457 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 17:08:14.072182 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.072166 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 17:08:14.081190 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.081165 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbvmj"] Jul 07 17:08:14.089910 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.089844 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4jnln"] Jul 07 17:08:14.152786 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.152755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-image-registry-private-configuration\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.152953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.152818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52dc686d-68c7-46e5-b4c0-8f622123e920-cert\") pod \"ingress-canary-wbvmj\" (UID: \"52dc686d-68c7-46e5-b4c0-8f622123e920\") " pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.152953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.152886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-ca-trust-extracted\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.153036 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.152951 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-certificates\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.153036 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.152976 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt7xp\" (UniqueName: \"kubernetes.io/projected/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-kube-api-access-pt7xp\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.153036 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-bound-sa-token\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.153171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153045 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-tmp-dir\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.153171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153073 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-installation-pull-secrets\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.153171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153092 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-config-volume\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.153277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153170 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns2nr\" (UniqueName: \"kubernetes.io/projected/52dc686d-68c7-46e5-b4c0-8f622123e920-kube-api-access-ns2nr\") pod \"ingress-canary-wbvmj\" (UID: \"52dc686d-68c7-46e5-b4c0-8f622123e920\") " pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.153277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.153277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153235 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fr8w\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-kube-api-access-5fr8w\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.153277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.153424 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.153338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-trusted-ca\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.254724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-certificates\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.254724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt7xp\" (UniqueName: \"kubernetes.io/projected/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-kube-api-access-pt7xp\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.254906 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254837 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-bound-sa-token\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.254906 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-tmp-dir\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.254976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-installation-pull-secrets\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.254976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-config-volume\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.255078 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.254985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns2nr\" (UniqueName: \"kubernetes.io/projected/52dc686d-68c7-46e5-b4c0-8f622123e920-kube-api-access-ns2nr\") pod \"ingress-canary-wbvmj\" (UID: \"52dc686d-68c7-46e5-b4c0-8f622123e920\") " pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.255078 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.255012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.255078 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.255034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fr8w\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-kube-api-access-5fr8w\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.255078 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.255057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.255259 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.255128 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:14.255259 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.255133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-trusted-ca\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.255259 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.255171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-image-registry-private-configuration\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.255259 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.255185 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:08:14.755166018 +0000 UTC m=+46.615246095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:14.255259 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.255196 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:14.255259 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.255210 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:14.255792 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.255769 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-certificates\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.256252 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.256059 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-config-volume\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.256252 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.256210 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-tmp-dir\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.256404 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.256328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52dc686d-68c7-46e5-b4c0-8f622123e920-cert\") pod \"ingress-canary-wbvmj\" (UID: \"52dc686d-68c7-46e5-b4c0-8f622123e920\") " pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.256404 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.256365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-ca-trust-extracted\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.256404 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.256377 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:08:14.756359771 +0000 UTC m=+46.616439835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:14.258694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.256831 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-ca-trust-extracted\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.258694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.257644 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-trusted-ca\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.260391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.260368 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-installation-pull-secrets\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.260687 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.260672 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52dc686d-68c7-46e5-b4c0-8f622123e920-cert\") pod \"ingress-canary-wbvmj\" (UID: \"52dc686d-68c7-46e5-b4c0-8f622123e920\") " pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.260749 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.260726 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-image-registry-private-configuration\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.270857 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.270834 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns2nr\" (UniqueName: \"kubernetes.io/projected/52dc686d-68c7-46e5-b4c0-8f622123e920-kube-api-access-ns2nr\") pod \"ingress-canary-wbvmj\" (UID: \"52dc686d-68c7-46e5-b4c0-8f622123e920\") " pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.271747 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.271708 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt7xp\" (UniqueName: \"kubernetes.io/projected/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-kube-api-access-pt7xp\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.273010 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.272991 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-bound-sa-token\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.275141 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.275122 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fr8w\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-kube-api-access-5fr8w\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.371756 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.371729 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbvmj" Jul 07 17:08:14.569074 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.569046 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbvmj"] Jul 07 17:08:14.572161 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:08:14.572136 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52dc686d_68c7_46e5_b4c0_8f622123e920.slice/crio-8fd8cd0056eed71b516be4b3f72539bcf247c9d4b6ee41c5beb52861409b11c8 WatchSource:0}: Error finding container 8fd8cd0056eed71b516be4b3f72539bcf247c9d4b6ee41c5beb52861409b11c8: Status 404 returned error can't find the container with id 8fd8cd0056eed71b516be4b3f72539bcf247c9d4b6ee41c5beb52861409b11c8 Jul 07 17:08:14.761068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.760985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:14.761068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.761028 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:14.761302 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.761098 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:14.761302 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.761123 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:14.761302 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.761201 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:08:15.761180775 +0000 UTC m=+47.621260846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:14.761302 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.761199 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:14.761302 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:14.761237 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:08:15.761228703 +0000 UTC m=+47.621308769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:14.761795 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.761768 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sj99_e1387f8c-3973-4bbf-8f83-a0d502302121/konnectivity-agent/0.log" limit=1024 Jul 07 17:08:14.848953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.848914 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sj99_e1387f8c-3973-4bbf-8f83-a0d502302121/konnectivity-agent/0.log" limit=1024 Jul 07 17:08:14.928062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.928023 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:14.928239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.928023 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:14.928239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.928028 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:14.931083 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.931060 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 17:08:14.931227 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.931198 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 17:08:14.931293 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.931201 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jk68m\"" Jul 07 17:08:14.931293 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.931240 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 17:08:14.931471 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.931458 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 17:08:14.932039 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:14.932022 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-chtgz\"" Jul 07 17:08:15.120811 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:15.120780 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbvmj" event={"ID":"52dc686d-68c7-46e5-b4c0-8f622123e920","Type":"ContainerStarted","Data":"8fd8cd0056eed71b516be4b3f72539bcf247c9d4b6ee41c5beb52861409b11c8"} Jul 07 17:08:15.768222 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:15.768185 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:15.768222 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:15.768223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:15.768466 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:15.768349 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:15.768466 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:15.768367 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:15.768466 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:15.768378 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:15.768466 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:15.768421 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:08:17.768406832 +0000 UTC m=+49.628486885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:15.768466 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:15.768435 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:08:17.768429183 +0000 UTC m=+49.628509232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:17.786564 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:17.786528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:17.786564 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:17.786566 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:17.787076 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:17.786696 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:17.787076 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:17.786706 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:17.787076 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:17.786732 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:17.787076 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:17.786758 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:08:21.786739221 +0000 UTC m=+53.646819290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:17.787076 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:17.786786 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:08:21.786768637 +0000 UTC m=+53.646848689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:18.346918 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:18.346887 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sj99_e1387f8c-3973-4bbf-8f83-a0d502302121/konnectivity-agent/0.log" limit=1024 Jul 07 17:08:18.449402 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:18.449364 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sj99_e1387f8c-3973-4bbf-8f83-a0d502302121/konnectivity-agent/0.log" limit=1024 Jul 07 17:08:19.130545 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:19.130504 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbvmj" event={"ID":"52dc686d-68c7-46e5-b4c0-8f622123e920","Type":"ContainerStarted","Data":"e49982db296f7bce8e64e08bd3eb9d79c4dbc5f84425b9236aaf2f9cf392c901"} Jul 07 17:08:19.153495 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:19.153442 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wbvmj" podStartSLOduration=1.392172028 podStartE2EDuration="5.153426772s" totalImagesPullingTime="3.761254744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:19.152661603 +0000 UTC m=+51.012741679" watchObservedRunningTime="2026-07-07 17:08:19.153426772 +0000 UTC m=+51.013506844" Jul 07 17:08:21.820441 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:21.820401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:21.820839 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:21.820492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:21.820839 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:21.820568 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:21.820839 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:21.820599 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:21.820839 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:21.820610 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:21.820839 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:21.820650 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:08:29.820637457 +0000 UTC m=+61.680717507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:21.820839 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:21.820662 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:08:29.820656015 +0000 UTC m=+61.680736065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:28.077542 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:28.077512 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f446s" Jul 07 17:08:29.872842 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:29.872796 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:29.872842 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:29.872839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:29.873271 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:29.872945 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:29.873271 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:29.872950 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:29.873271 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:29.872971 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:29.873271 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:29.873006 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:08:45.872988151 +0000 UTC m=+77.733068201 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:29.873271 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:29.873020 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:08:45.873013763 +0000 UTC m=+77.733093812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:34.503802 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.503699 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:08:34.503802 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.503747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:34.506137 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.506115 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 17:08:34.506238 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.506223 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 17:08:34.514122 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:34.514102 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 17:08:34.514238 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:34.514175 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs podName:158c3d23-9652-4f2b-acee-d535a4f662f6 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:38.514158155 +0000 UTC m=+130.374238205 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs") pod "network-metrics-daemon-7n5g9" (UID: "158c3d23-9652-4f2b-acee-d535a4f662f6") : secret "metrics-daemon-secret" not found Jul 07 17:08:34.516288 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.516261 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/039a7f67-f570-4358-8404-083823f9c823-original-pull-secret\") pod \"global-pull-secret-syncer-442hr\" (UID: \"039a7f67-f570-4358-8404-083823f9c823\") " pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:34.604407 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.604372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:34.607317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.607297 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 17:08:34.617891 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.617869 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 17:08:34.628400 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.628368 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62jc\" (UniqueName: \"kubernetes.io/projected/c3af20bb-2a20-4a61-87e3-fd19895e3a70-kube-api-access-b62jc\") pod \"network-check-target-qqgwn\" (UID: \"c3af20bb-2a20-4a61-87e3-fd19895e3a70\") " pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:34.741084 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.741056 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jk68m\"" Jul 07 17:08:34.744218 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.744200 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-442hr" Jul 07 17:08:34.748422 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.748402 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:34.924734 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.924699 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-442hr"] Jul 07 17:08:34.928381 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:08:34.928345 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039a7f67_f570_4358_8404_083823f9c823.slice/crio-844e9ade51fc84508fd064011ed4eaa1a47ecda5866b43fb010f84df71d5ec53 WatchSource:0}: Error finding container 844e9ade51fc84508fd064011ed4eaa1a47ecda5866b43fb010f84df71d5ec53: Status 404 returned error can't find the container with id 844e9ade51fc84508fd064011ed4eaa1a47ecda5866b43fb010f84df71d5ec53 Jul 07 17:08:34.941640 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:34.941605 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qqgwn"] Jul 07 17:08:34.944522 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:08:34.944499 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3af20bb_2a20_4a61_87e3_fd19895e3a70.slice/crio-7fcb4e4500395238aba996ecf8a26c44efd6e66a859d7fc04bfcacc0d18f8558 WatchSource:0}: Error finding container 7fcb4e4500395238aba996ecf8a26c44efd6e66a859d7fc04bfcacc0d18f8558: Status 404 returned error can't find the container with id 7fcb4e4500395238aba996ecf8a26c44efd6e66a859d7fc04bfcacc0d18f8558 Jul 07 17:08:35.168990 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:35.168950 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qqgwn" event={"ID":"c3af20bb-2a20-4a61-87e3-fd19895e3a70","Type":"ContainerStarted","Data":"7fcb4e4500395238aba996ecf8a26c44efd6e66a859d7fc04bfcacc0d18f8558"} Jul 07 17:08:35.169894 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:35.169865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-442hr" event={"ID":"039a7f67-f570-4358-8404-083823f9c823","Type":"ContainerStarted","Data":"844e9ade51fc84508fd064011ed4eaa1a47ecda5866b43fb010f84df71d5ec53"} Jul 07 17:08:43.194172 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:43.194134 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qqgwn" event={"ID":"c3af20bb-2a20-4a61-87e3-fd19895e3a70","Type":"ContainerStarted","Data":"39c7faa8cc04bf29c9ee53236a733e9a805b2a7d3a04cd2c174f6a7af64a3be1"} Jul 07 17:08:43.194652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:43.194339 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:08:43.195507 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:43.195485 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-442hr" event={"ID":"039a7f67-f570-4358-8404-083823f9c823","Type":"ContainerStarted","Data":"08754619934c6819e3162f4900e3a7acfdcb48eaebbfa8af036a9b27203f80ee"} Jul 07 17:08:43.215869 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:43.215828 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qqgwn" podStartSLOduration=66.639325406 podStartE2EDuration="1m14.215815451s" totalImagesPullingTime="7.576490045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:43.213810281 +0000 UTC m=+75.073890350" watchObservedRunningTime="2026-07-07 17:08:43.215815451 +0000 UTC m=+75.075895523" Jul 07 17:08:43.233874 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:43.233838 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-442hr" podStartSLOduration=66.605895807 podStartE2EDuration="1m14.233825733s" totalImagesPullingTime="7.627929926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:43.23375093 +0000 UTC m=+75.093831003" watchObservedRunningTime="2026-07-07 17:08:43.233825733 +0000 UTC m=+75.093905806" Jul 07 17:08:45.888706 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:45.888669 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:08:45.888706 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:45.888709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:08:45.889114 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:45.888833 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:08:45.889114 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:45.888839 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:08:45.889114 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:45.888858 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67447d8bcd-fqdlm: secret "image-registry-tls" not found Jul 07 17:08:45.889114 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:45.888887 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls podName:7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b nodeName:}" failed. No retries permitted until 2026-07-07 17:09:17.888873654 +0000 UTC m=+109.748953704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls") pod "dns-default-4jnln" (UID: "7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b") : secret "dns-default-metrics-tls" not found Jul 07 17:08:45.889114 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:45.888915 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls podName:e59b48ea-ba2f-4eb5-b74c-1f246b8465fb nodeName:}" failed. No retries permitted until 2026-07-07 17:09:17.888901499 +0000 UTC m=+109.748981553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls") pod "image-registry-67447d8bcd-fqdlm" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb") : secret "image-registry-tls" not found Jul 07 17:08:47.349116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.349083 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc"] Jul 07 17:08:47.352062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.352037 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" Jul 07 17:08:47.353031 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.353011 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-frwl4"] Jul 07 17:08:47.355321 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.355306 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-54r2j\"" Jul 07 17:08:47.355587 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.355558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.360536 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.360518 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 17:08:47.360655 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.360519 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 17:08:47.360655 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.360557 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 17:08:47.361495 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.361479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:08:47.361597 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.361491 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-q8vs4\"" Jul 07 17:08:47.364866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.364772 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc"] Jul 07 17:08:47.367640 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.367619 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 17:08:47.372157 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.372129 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-frwl4"] Jul 07 17:08:47.398465 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.398441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/346488bd-0761-4f1f-a947-c476615e2deb-serving-cert\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.398627 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.398471 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/346488bd-0761-4f1f-a947-c476615e2deb-trusted-ca\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.398627 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.398495 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zldbt\" (UniqueName: \"kubernetes.io/projected/b087cbb2-0219-401a-917a-31adf926b99b-kube-api-access-zldbt\") pod \"network-check-source-989f8b8dc-7fpsc\" (UID: \"b087cbb2-0219-401a-917a-31adf926b99b\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" Jul 07 17:08:47.398627 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.398512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/346488bd-0761-4f1f-a947-c476615e2deb-config\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.398627 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.398569 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbwbg\" (UniqueName: \"kubernetes.io/projected/346488bd-0761-4f1f-a947-c476615e2deb-kube-api-access-pbwbg\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.478523 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.478489 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf"] Jul 07 17:08:47.481388 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.481372 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.485346 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.484390 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fxszj\"" Jul 07 17:08:47.485346 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.484478 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:08:47.485346 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.485097 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 17:08:47.485590 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.485433 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 17:08:47.485656 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.485636 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 17:08:47.499224 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.499197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/346488bd-0761-4f1f-a947-c476615e2deb-serving-cert\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.499362 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.499239 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/346488bd-0761-4f1f-a947-c476615e2deb-trusted-ca\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.499362 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.499271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zldbt\" (UniqueName: \"kubernetes.io/projected/b087cbb2-0219-401a-917a-31adf926b99b-kube-api-access-zldbt\") pod \"network-check-source-989f8b8dc-7fpsc\" (UID: \"b087cbb2-0219-401a-917a-31adf926b99b\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" Jul 07 17:08:47.499498 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.499431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/346488bd-0761-4f1f-a947-c476615e2deb-config\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.499498 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.499490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbwbg\" (UniqueName: \"kubernetes.io/projected/346488bd-0761-4f1f-a947-c476615e2deb-kube-api-access-pbwbg\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.500210 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.500180 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/346488bd-0761-4f1f-a947-c476615e2deb-config\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.500375 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.500357 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/346488bd-0761-4f1f-a947-c476615e2deb-trusted-ca\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.502056 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.502036 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/346488bd-0761-4f1f-a947-c476615e2deb-serving-cert\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.502684 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.502668 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz"] Jul 07 17:08:47.505583 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.505559 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf"] Jul 07 17:08:47.505689 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.505668 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:47.508521 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.508502 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 17:08:47.509289 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.509271 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:08:47.509357 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.509344 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 17:08:47.510065 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.510050 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8zchl\"" Jul 07 17:08:47.517539 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.517517 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbwbg\" (UniqueName: \"kubernetes.io/projected/346488bd-0761-4f1f-a947-c476615e2deb-kube-api-access-pbwbg\") pod \"console-operator-68b757fb67-frwl4\" (UID: \"346488bd-0761-4f1f-a947-c476615e2deb\") " pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.521533 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.521514 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zldbt\" (UniqueName: \"kubernetes.io/projected/b087cbb2-0219-401a-917a-31adf926b99b-kube-api-access-zldbt\") pod \"network-check-source-989f8b8dc-7fpsc\" (UID: \"b087cbb2-0219-401a-917a-31adf926b99b\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" Jul 07 17:08:47.531859 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.531835 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz"] Jul 07 17:08:47.600917 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.600827 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8v5q\" (UniqueName: \"kubernetes.io/projected/b3ece572-741a-4ca7-8474-9a5e8c897ed3-kube-api-access-c8v5q\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:47.600917 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.600874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:47.601102 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.600934 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d983025a-e880-4d95-a161-da213fda040b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.601102 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.601001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d983025a-e880-4d95-a161-da213fda040b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.601102 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.601021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkvkt\" (UniqueName: \"kubernetes.io/projected/d983025a-e880-4d95-a161-da213fda040b-kube-api-access-qkvkt\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.662683 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.662646 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" Jul 07 17:08:47.667723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.667700 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:47.701667 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.701622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8v5q\" (UniqueName: \"kubernetes.io/projected/b3ece572-741a-4ca7-8474-9a5e8c897ed3-kube-api-access-c8v5q\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:47.701667 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.701665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:47.701933 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.701693 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d983025a-e880-4d95-a161-da213fda040b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.701933 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.701742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d983025a-e880-4d95-a161-da213fda040b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.701933 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.701759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkvkt\" (UniqueName: \"kubernetes.io/projected/d983025a-e880-4d95-a161-da213fda040b-kube-api-access-qkvkt\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.701933 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:47.701796 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:08:47.701933 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:47.701870 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls podName:b3ece572-741a-4ca7-8474-9a5e8c897ed3 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:48.201848661 +0000 UTC m=+80.061928730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-76wtz" (UID: "b3ece572-741a-4ca7-8474-9a5e8c897ed3") : secret "samples-operator-tls" not found Jul 07 17:08:47.702386 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.702365 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d983025a-e880-4d95-a161-da213fda040b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.703950 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.703933 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d983025a-e880-4d95-a161-da213fda040b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.718964 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.718807 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8v5q\" (UniqueName: \"kubernetes.io/projected/b3ece572-741a-4ca7-8474-9a5e8c897ed3-kube-api-access-c8v5q\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:47.719396 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.719261 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkvkt\" (UniqueName: \"kubernetes.io/projected/d983025a-e880-4d95-a161-da213fda040b-kube-api-access-qkvkt\") pod \"kube-storage-version-migrator-operator-846bf668f7-6x5tf\" (UID: \"d983025a-e880-4d95-a161-da213fda040b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.791974 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.791942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" Jul 07 17:08:47.797227 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.797197 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc"] Jul 07 17:08:47.801544 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:08:47.801514 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb087cbb2_0219_401a_917a_31adf926b99b.slice/crio-d3843adb40dfb198a98a2580210aa30967b6d31012aa9a3f1ef6d008c8da8432 WatchSource:0}: Error finding container d3843adb40dfb198a98a2580210aa30967b6d31012aa9a3f1ef6d008c8da8432: Status 404 returned error can't find the container with id d3843adb40dfb198a98a2580210aa30967b6d31012aa9a3f1ef6d008c8da8432 Jul 07 17:08:47.813653 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.813592 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-frwl4"] Jul 07 17:08:47.817722 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:08:47.817683 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346488bd_0761_4f1f_a947_c476615e2deb.slice/crio-408c89aad84d867665fbe21592bd3930e91a05143fb02b772af501b36488ecec WatchSource:0}: Error finding container 408c89aad84d867665fbe21592bd3930e91a05143fb02b772af501b36488ecec: Status 404 returned error can't find the container with id 408c89aad84d867665fbe21592bd3930e91a05143fb02b772af501b36488ecec Jul 07 17:08:47.926871 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:47.926839 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf"] Jul 07 17:08:47.929788 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:08:47.929763 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd983025a_e880_4d95_a161_da213fda040b.slice/crio-1a985b71277f35ab88b9d9885f7f5e37541d3a9f1bcdee1d7ca3fdfc05a7ad3d WatchSource:0}: Error finding container 1a985b71277f35ab88b9d9885f7f5e37541d3a9f1bcdee1d7ca3fdfc05a7ad3d: Status 404 returned error can't find the container with id 1a985b71277f35ab88b9d9885f7f5e37541d3a9f1bcdee1d7ca3fdfc05a7ad3d Jul 07 17:08:48.205254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:48.205221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:48.205431 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:48.205380 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:08:48.205488 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:48.205451 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls podName:b3ece572-741a-4ca7-8474-9a5e8c897ed3 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:49.205435308 +0000 UTC m=+81.065515359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-76wtz" (UID: "b3ece572-741a-4ca7-8474-9a5e8c897ed3") : secret "samples-operator-tls" not found Jul 07 17:08:48.208647 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:48.208614 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" event={"ID":"d983025a-e880-4d95-a161-da213fda040b","Type":"ContainerStarted","Data":"1a985b71277f35ab88b9d9885f7f5e37541d3a9f1bcdee1d7ca3fdfc05a7ad3d"} Jul 07 17:08:48.209645 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:48.209625 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" event={"ID":"346488bd-0761-4f1f-a947-c476615e2deb","Type":"ContainerStarted","Data":"408c89aad84d867665fbe21592bd3930e91a05143fb02b772af501b36488ecec"} Jul 07 17:08:48.211082 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:48.211061 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" event={"ID":"b087cbb2-0219-401a-917a-31adf926b99b","Type":"ContainerStarted","Data":"7fb61c2fbd4e677d00440ef65f78912cc38f7f6b86da5ef3ae8d23d43c769b2f"} Jul 07 17:08:48.211170 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:48.211104 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" event={"ID":"b087cbb2-0219-401a-917a-31adf926b99b","Type":"ContainerStarted","Data":"d3843adb40dfb198a98a2580210aa30967b6d31012aa9a3f1ef6d008c8da8432"} Jul 07 17:08:48.231336 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:48.231290 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7fpsc" podStartSLOduration=1.231278406 podStartE2EDuration="1.231278406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:48.230886961 +0000 UTC m=+80.090967033" watchObservedRunningTime="2026-07-07 17:08:48.231278406 +0000 UTC m=+80.091358516" Jul 07 17:08:49.214013 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:49.213962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:49.220048 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:49.214637 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:08:49.220048 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:49.214723 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls podName:b3ece572-741a-4ca7-8474-9a5e8c897ed3 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:51.21470337 +0000 UTC m=+83.074783422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-76wtz" (UID: "b3ece572-741a-4ca7-8474-9a5e8c897ed3") : secret "samples-operator-tls" not found Jul 07 17:08:51.223629 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.223478 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/0.log" Jul 07 17:08:51.223629 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.223528 2582 generic.go:356] "Generic (PLEG): container finished" podID="346488bd-0761-4f1f-a947-c476615e2deb" containerID="e55dd6750b0f964d162b7a74407cabee87032e91ea4ebb2314bcae8a6377effb" exitCode=255 Jul 07 17:08:51.223629 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.223597 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" event={"ID":"346488bd-0761-4f1f-a947-c476615e2deb","Type":"ContainerDied","Data":"e55dd6750b0f964d162b7a74407cabee87032e91ea4ebb2314bcae8a6377effb"} Jul 07 17:08:51.224173 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.223926 2582 scope.go:117] "RemoveContainer" containerID="e55dd6750b0f964d162b7a74407cabee87032e91ea4ebb2314bcae8a6377effb" Jul 07 17:08:51.233407 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.233380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:51.233544 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:51.233521 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:08:51.233665 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:51.233608 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls podName:b3ece572-741a-4ca7-8474-9a5e8c897ed3 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:55.233588496 +0000 UTC m=+87.093668570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-76wtz" (UID: "b3ece572-741a-4ca7-8474-9a5e8c897ed3") : secret "samples-operator-tls" not found Jul 07 17:08:51.563814 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.563731 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-txwgh"] Jul 07 17:08:51.566893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.566874 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.569586 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.569553 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 17:08:51.570104 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.570074 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 17:08:51.570104 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.570094 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 17:08:51.570452 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.570436 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 17:08:51.570805 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.570789 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kb2xx\"" Jul 07 17:08:51.581195 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.581168 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-txwgh"] Jul 07 17:08:51.636512 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.636476 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.636512 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.636511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.636736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.636557 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbc5b\" (UniqueName: \"kubernetes.io/projected/28c670d9-ad00-4f66-a082-613c023982f5-kube-api-access-vbc5b\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.636736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.636597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28c670d9-ad00-4f66-a082-613c023982f5-data-volume\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.636736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.636626 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28c670d9-ad00-4f66-a082-613c023982f5-crio-socket\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737481 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737481 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737715 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbc5b\" (UniqueName: \"kubernetes.io/projected/28c670d9-ad00-4f66-a082-613c023982f5-kube-api-access-vbc5b\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737715 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:51.737613 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:08:51.737715 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28c670d9-ad00-4f66-a082-613c023982f5-data-volume\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737715 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:51.737615 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:08:51.737715 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:51.737686 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:52.237666415 +0000 UTC m=+84.097746496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : configmap "kube-rbac-proxy" not found Jul 07 17:08:51.737715 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28c670d9-ad00-4f66-a082-613c023982f5-crio-socket\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737988 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:51.737729 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:52.237710076 +0000 UTC m=+84.097790139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : secret "insights-runtime-extractor-tls" not found Jul 07 17:08:51.737988 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737760 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28c670d9-ad00-4f66-a082-613c023982f5-crio-socket\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.737988 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.737928 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28c670d9-ad00-4f66-a082-613c023982f5-data-volume\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:51.751207 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:51.751184 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbc5b\" (UniqueName: \"kubernetes.io/projected/28c670d9-ad00-4f66-a082-613c023982f5-kube-api-access-vbc5b\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:52.229403 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.229365 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" event={"ID":"d983025a-e880-4d95-a161-da213fda040b","Type":"ContainerStarted","Data":"a35f247a1d07754de08d37dc729685a03d9ca294ffdd330d1f48ce4ea1138936"} Jul 07 17:08:52.230871 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.230853 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/1.log" Jul 07 17:08:52.231241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.231228 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/0.log" Jul 07 17:08:52.231297 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.231257 2582 generic.go:356] "Generic (PLEG): container finished" podID="346488bd-0761-4f1f-a947-c476615e2deb" containerID="044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504" exitCode=255 Jul 07 17:08:52.231297 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.231280 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" event={"ID":"346488bd-0761-4f1f-a947-c476615e2deb","Type":"ContainerDied","Data":"044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504"} Jul 07 17:08:52.231388 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.231329 2582 scope.go:117] "RemoveContainer" containerID="e55dd6750b0f964d162b7a74407cabee87032e91ea4ebb2314bcae8a6377effb" Jul 07 17:08:52.231533 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.231519 2582 scope.go:117] "RemoveContainer" containerID="044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504" Jul 07 17:08:52.231742 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:52.231720 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-frwl4_openshift-console-operator(346488bd-0761-4f1f-a947-c476615e2deb)\"" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" podUID="346488bd-0761-4f1f-a947-c476615e2deb" Jul 07 17:08:52.241958 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.241937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:52.242046 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.241968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:52.242046 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:52.242025 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:08:52.242110 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:52.242068 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:53.242055844 +0000 UTC m=+85.102135899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : configmap "kube-rbac-proxy" not found Jul 07 17:08:52.242110 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:52.242067 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:08:52.242174 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:52.242124 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:53.242104988 +0000 UTC m=+85.102185041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : secret "insights-runtime-extractor-tls" not found Jul 07 17:08:52.251114 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:52.251073 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" podStartSLOduration=1.829287135 podStartE2EDuration="5.251061012s" totalImagesPullingTime="3.421773877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:52.249820409 +0000 UTC m=+84.109900484" watchObservedRunningTime="2026-07-07 17:08:52.251061012 +0000 UTC m=+84.111141083" Jul 07 17:08:53.235604 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:53.235560 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/1.log" Jul 07 17:08:53.235985 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:53.235902 2582 scope.go:117] "RemoveContainer" containerID="044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504" Jul 07 17:08:53.236073 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:53.236053 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-frwl4_openshift-console-operator(346488bd-0761-4f1f-a947-c476615e2deb)\"" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" podUID="346488bd-0761-4f1f-a947-c476615e2deb" Jul 07 17:08:53.249892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:53.249870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:53.249994 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:53.249899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:53.249994 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:53.249984 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:08:53.250075 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:53.249994 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:08:53.250075 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:53.250034 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:55.250017194 +0000 UTC m=+87.110097245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : configmap "kube-rbac-proxy" not found Jul 07 17:08:53.250075 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:53.250048 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:55.250041874 +0000 UTC m=+87.110121924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : secret "insights-runtime-extractor-tls" not found Jul 07 17:08:53.769959 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:53.769927 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tqtx4_3ff53b6a-4006-4d81-88c4-6cae9a6590ef/dns-node-resolver/0.log" Jul 07 17:08:54.570113 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:54.570083 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f9gq4_db8bf3da-6be2-460d-b95d-004822f9a091/node-ca/0.log" Jul 07 17:08:55.269723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:55.269680 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:55.269723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:55.269727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:55.269807 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:55.269830 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:55.269883 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:55.269888 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:59.269873481 +0000 UTC m=+91.129953531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : secret "insights-runtime-extractor-tls" not found Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:55.269919 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls podName:b3ece572-741a-4ca7-8474-9a5e8c897ed3 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:03.269908916 +0000 UTC m=+95.129988967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-76wtz" (UID: "b3ece572-741a-4ca7-8474-9a5e8c897ed3") : secret "samples-operator-tls" not found Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:55.269878 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:08:55.269984 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:55.269945 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:59.269939212 +0000 UTC m=+91.130019262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : configmap "kube-rbac-proxy" not found Jul 07 17:08:55.572124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:55.572048 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbvmj_52dc686d-68c7-46e5-b4c0-8f622123e920/serve-healthcheck-canary/0.log" Jul 07 17:08:57.667938 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:57.667893 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:57.668305 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:57.667982 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:08:57.668341 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:57.668308 2582 scope.go:117] "RemoveContainer" containerID="044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504" Jul 07 17:08:57.668482 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:57.668461 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-frwl4_openshift-console-operator(346488bd-0761-4f1f-a947-c476615e2deb)\"" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" podUID="346488bd-0761-4f1f-a947-c476615e2deb" Jul 07 17:08:59.304824 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:59.304785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:59.304824 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:08:59.304825 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:08:59.305315 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:59.304956 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:08:59.305315 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:59.304978 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:08:59.305315 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:59.305038 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:07.305020433 +0000 UTC m=+99.165100496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : configmap "kube-rbac-proxy" not found Jul 07 17:08:59.305315 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:08:59.305052 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls podName:28c670d9-ad00-4f66-a082-613c023982f5 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:07.305045784 +0000 UTC m=+99.165125834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txwgh" (UID: "28c670d9-ad00-4f66-a082-613c023982f5") : secret "insights-runtime-extractor-tls" not found Jul 07 17:09:03.340543 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:03.340502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:09:03.342875 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:03.342854 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3ece572-741a-4ca7-8474-9a5e8c897ed3-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-76wtz\" (UID: \"b3ece572-741a-4ca7-8474-9a5e8c897ed3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:09:03.414955 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:03.414925 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" Jul 07 17:09:03.554941 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:03.554915 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz"] Jul 07 17:09:04.271437 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:04.271400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" event={"ID":"b3ece572-741a-4ca7-8474-9a5e8c897ed3","Type":"ContainerStarted","Data":"e9fee01839cd680073661b13b2977f3890da075ea81a536dd2480b37aa6a3529"} Jul 07 17:09:06.278855 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:06.278819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" event={"ID":"b3ece572-741a-4ca7-8474-9a5e8c897ed3","Type":"ContainerStarted","Data":"6b5633467acaed77047b0fe1d09c93054b451d289c571c57aea78c08c550d2a1"} Jul 07 17:09:06.279273 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:06.278885 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" event={"ID":"b3ece572-741a-4ca7-8474-9a5e8c897ed3","Type":"ContainerStarted","Data":"9cfee42a1ad1f2370d9cc693551013959e1b4519493054eab279ffbfeaef8980"} Jul 07 17:09:06.303752 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:06.303665 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-76wtz" podStartSLOduration=16.856567892 podStartE2EDuration="19.30365145s" totalImagesPullingTime="2.447083558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:06.302204372 +0000 UTC m=+98.162284445" watchObservedRunningTime="2026-07-07 17:09:06.30365145 +0000 UTC m=+98.163731521" Jul 07 17:09:07.375237 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:07.375195 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:09:07.375237 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:07.375237 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:09:07.375775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:07.375749 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28c670d9-ad00-4f66-a082-613c023982f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:09:07.377644 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:07.377614 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28c670d9-ad00-4f66-a082-613c023982f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txwgh\" (UID: \"28c670d9-ad00-4f66-a082-613c023982f5\") " pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:09:07.476054 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:07.476007 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-txwgh" Jul 07 17:09:07.604257 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:07.604227 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-txwgh"] Jul 07 17:09:07.607276 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:07.607251 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c670d9_ad00_4f66_a082_613c023982f5.slice/crio-c42cfe18a1d9b82ae87b801530be01bb1905ef086ae4bc8cfe78c36dabdc2af9 WatchSource:0}: Error finding container c42cfe18a1d9b82ae87b801530be01bb1905ef086ae4bc8cfe78c36dabdc2af9: Status 404 returned error can't find the container with id c42cfe18a1d9b82ae87b801530be01bb1905ef086ae4bc8cfe78c36dabdc2af9 Jul 07 17:09:08.289774 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:08.289734 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txwgh" event={"ID":"28c670d9-ad00-4f66-a082-613c023982f5","Type":"ContainerStarted","Data":"c42cfe18a1d9b82ae87b801530be01bb1905ef086ae4bc8cfe78c36dabdc2af9"} Jul 07 17:09:08.930564 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:08.930529 2582 scope.go:117] "RemoveContainer" containerID="044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504" Jul 07 17:09:09.294746 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:09.294672 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:09:09.295115 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:09.295098 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/1.log" Jul 07 17:09:09.295214 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:09.295134 2582 generic.go:356] "Generic (PLEG): container finished" podID="346488bd-0761-4f1f-a947-c476615e2deb" containerID="35444812c57cb1f4c6bac8a146e6922913bf1864b47a664b5a015c66844095d0" exitCode=255 Jul 07 17:09:09.295214 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:09.295203 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" event={"ID":"346488bd-0761-4f1f-a947-c476615e2deb","Type":"ContainerDied","Data":"35444812c57cb1f4c6bac8a146e6922913bf1864b47a664b5a015c66844095d0"} Jul 07 17:09:09.295294 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:09.295265 2582 scope.go:117] "RemoveContainer" containerID="044250328f1267bb19e01d19141f6a06d5abeb111bbd07df6fe12600087aa504" Jul 07 17:09:09.295652 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:09.295628 2582 scope.go:117] "RemoveContainer" containerID="35444812c57cb1f4c6bac8a146e6922913bf1864b47a664b5a015c66844095d0" Jul 07 17:09:09.295856 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:09:09.295836 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-frwl4_openshift-console-operator(346488bd-0761-4f1f-a947-c476615e2deb)\"" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" podUID="346488bd-0761-4f1f-a947-c476615e2deb" Jul 07 17:09:12.307422 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:12.307396 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:09:12.309116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:12.309090 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txwgh" event={"ID":"28c670d9-ad00-4f66-a082-613c023982f5","Type":"ContainerStarted","Data":"08a1f4a9d1e52a4c046e39dd42d437d677f89875703d2262396d94ed059cbff3"} Jul 07 17:09:12.309232 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:12.309143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txwgh" event={"ID":"28c670d9-ad00-4f66-a082-613c023982f5","Type":"ContainerStarted","Data":"841e6903d43c54a051137ef65fe6d9d5de44758c72d6ba9d76f32e8e50bb6ec2"} Jul 07 17:09:14.201382 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:14.201346 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qqgwn" Jul 07 17:09:16.326188 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:16.326156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txwgh" event={"ID":"28c670d9-ad00-4f66-a082-613c023982f5","Type":"ContainerStarted","Data":"ed5e56f9f26d23f755c5922240a6d4edfcb88bdd82ead6364be519500601c721"} Jul 07 17:09:16.349502 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:16.349453 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-txwgh" podStartSLOduration=17.778274164 podStartE2EDuration="25.349436778s" totalImagesPullingTime="7.571162614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:08:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:16.348697604 +0000 UTC m=+108.208777680" watchObservedRunningTime="2026-07-07 17:09:16.349436778 +0000 UTC m=+108.209516850" Jul 07 17:09:17.668066 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.668020 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:09:17.668680 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.668093 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:09:17.668680 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.668438 2582 scope.go:117] "RemoveContainer" containerID="35444812c57cb1f4c6bac8a146e6922913bf1864b47a664b5a015c66844095d0" Jul 07 17:09:17.668680 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:09:17.668657 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-frwl4_openshift-console-operator(346488bd-0761-4f1f-a947-c476615e2deb)\"" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" podUID="346488bd-0761-4f1f-a947-c476615e2deb" Jul 07 17:09:17.964211 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.964117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:09:17.964389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.964214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:09:17.966629 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.966603 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b-metrics-tls\") pod \"dns-default-4jnln\" (UID: \"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b\") " pod="openshift-dns/dns-default-4jnln" Jul 07 17:09:17.966629 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:17.966618 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"image-registry-67447d8bcd-fqdlm\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:09:18.247904 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:18.247823 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n59wn\"" Jul 07 17:09:18.256038 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:18.256008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:09:18.267646 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:18.267612 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r2xr\"" Jul 07 17:09:18.276166 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:18.276143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4jnln" Jul 07 17:09:18.397146 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:18.397125 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67447d8bcd-fqdlm"] Jul 07 17:09:18.399438 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:18.399417 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59b48ea_ba2f_4eb5_b74c_1f246b8465fb.slice/crio-51c763448008e6095f7c5d32186b0cce034d8d9eba4aa8894e67ac518dc9e386 WatchSource:0}: Error finding container 51c763448008e6095f7c5d32186b0cce034d8d9eba4aa8894e67ac518dc9e386: Status 404 returned error can't find the container with id 51c763448008e6095f7c5d32186b0cce034d8d9eba4aa8894e67ac518dc9e386 Jul 07 17:09:18.416604 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:18.416567 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4jnln"] Jul 07 17:09:18.419322 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:18.419302 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7385a2_6dfd_41d7_a6e6_2c05f8ce5c4b.slice/crio-418b4e037a5a03b738fb03700ba012992665b287849a797e74d18f98006acc3a WatchSource:0}: Error finding container 418b4e037a5a03b738fb03700ba012992665b287849a797e74d18f98006acc3a: Status 404 returned error can't find the container with id 418b4e037a5a03b738fb03700ba012992665b287849a797e74d18f98006acc3a Jul 07 17:09:19.336797 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:19.336752 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4jnln" event={"ID":"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b","Type":"ContainerStarted","Data":"418b4e037a5a03b738fb03700ba012992665b287849a797e74d18f98006acc3a"} Jul 07 17:09:19.338058 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:19.338027 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" event={"ID":"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb","Type":"ContainerStarted","Data":"b3ce3ea9c7f9802bd91b22361baeaa1e2444e48519303a64423a68d92f6f489b"} Jul 07 17:09:19.338214 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:19.338074 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" event={"ID":"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb","Type":"ContainerStarted","Data":"51c763448008e6095f7c5d32186b0cce034d8d9eba4aa8894e67ac518dc9e386"} Jul 07 17:09:19.338214 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:19.338151 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:09:19.361582 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:19.361522 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" podStartSLOduration=110.3615054 podStartE2EDuration="1m50.3615054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:19.359917343 +0000 UTC m=+111.219997414" watchObservedRunningTime="2026-07-07 17:09:19.3615054 +0000 UTC m=+111.221585501" Jul 07 17:09:20.343399 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.343372 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4jnln" event={"ID":"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b","Type":"ContainerStarted","Data":"3a5172584a79ba554b7fdf62b810e421a2ad56379b5b099049ed486a2fc6367d"} Jul 07 17:09:20.856592 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.856551 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5"] Jul 07 17:09:20.859817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.859799 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.863260 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.863236 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 17:09:20.863383 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.863292 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 17:09:20.865051 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.865032 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 17:09:20.865233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.865221 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 17:09:20.878421 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.878401 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5"] Jul 07 17:09:20.886759 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.886736 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b55b54c6-1aec-4c5b-ab0c-50867b607552-tmp\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.886864 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.886812 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbb8p\" (UniqueName: \"kubernetes.io/projected/b55b54c6-1aec-4c5b-ab0c-50867b607552-kube-api-access-cbb8p\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.886864 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.886842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b55b54c6-1aec-4c5b-ab0c-50867b607552-klusterlet-config\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.948219 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.948186 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-446rp"] Jul 07 17:09:20.951049 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.951034 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:20.960727 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.960706 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 17:09:20.961754 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.961735 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 17:09:20.963687 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.963673 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-h5tmn\"" Jul 07 17:09:20.980207 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.977350 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-446rp"] Jul 07 17:09:20.987189 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.987165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/70f4fece-813b-4fff-aa86-06d7c1ca0575-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-446rp\" (UID: \"70f4fece-813b-4fff-aa86-06d7c1ca0575\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:20.987277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.987200 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b55b54c6-1aec-4c5b-ab0c-50867b607552-klusterlet-config\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.987277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.987228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b55b54c6-1aec-4c5b-ab0c-50867b607552-tmp\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.987277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.987258 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70f4fece-813b-4fff-aa86-06d7c1ca0575-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-446rp\" (UID: \"70f4fece-813b-4fff-aa86-06d7c1ca0575\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:20.987428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.987310 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbb8p\" (UniqueName: \"kubernetes.io/projected/b55b54c6-1aec-4c5b-ab0c-50867b607552-kube-api-access-cbb8p\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.987596 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.987559 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b55b54c6-1aec-4c5b-ab0c-50867b607552-tmp\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:20.989873 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:20.989851 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b55b54c6-1aec-4c5b-ab0c-50867b607552-klusterlet-config\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:21.012854 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.012828 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbb8p\" (UniqueName: \"kubernetes.io/projected/b55b54c6-1aec-4c5b-ab0c-50867b607552-kube-api-access-cbb8p\") pod \"klusterlet-addon-workmgr-55d7b667b7-rnhq5\" (UID: \"b55b54c6-1aec-4c5b-ab0c-50867b607552\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:21.088605 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.088544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/70f4fece-813b-4fff-aa86-06d7c1ca0575-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-446rp\" (UID: \"70f4fece-813b-4fff-aa86-06d7c1ca0575\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:21.088799 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.088648 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70f4fece-813b-4fff-aa86-06d7c1ca0575-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-446rp\" (UID: \"70f4fece-813b-4fff-aa86-06d7c1ca0575\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:21.089179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.089154 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/70f4fece-813b-4fff-aa86-06d7c1ca0575-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-446rp\" (UID: \"70f4fece-813b-4fff-aa86-06d7c1ca0575\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:21.091031 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.091008 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70f4fece-813b-4fff-aa86-06d7c1ca0575-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-446rp\" (UID: \"70f4fece-813b-4fff-aa86-06d7c1ca0575\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:21.168525 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.168490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:21.260179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.260147 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" Jul 07 17:09:21.292495 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.292463 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5"] Jul 07 17:09:21.295352 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:21.295318 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55b54c6_1aec_4c5b_ab0c_50867b607552.slice/crio-92fbc7420e21398e5749767d839c7d8b740d0db065a83928f53f2945e176f30c WatchSource:0}: Error finding container 92fbc7420e21398e5749767d839c7d8b740d0db065a83928f53f2945e176f30c: Status 404 returned error can't find the container with id 92fbc7420e21398e5749767d839c7d8b740d0db065a83928f53f2945e176f30c Jul 07 17:09:21.370782 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.370749 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" event={"ID":"b55b54c6-1aec-4c5b-ab0c-50867b607552","Type":"ContainerStarted","Data":"92fbc7420e21398e5749767d839c7d8b740d0db065a83928f53f2945e176f30c"} Jul 07 17:09:21.372687 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.372653 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4jnln" event={"ID":"7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b","Type":"ContainerStarted","Data":"923478cd390aaa04c1e9ca41d0a5fbe686c0fe46847c7a7327f6c4ba0b5cfd58"} Jul 07 17:09:21.372822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.372776 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4jnln" Jul 07 17:09:21.393401 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.392894 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4jnln" podStartSLOduration=65.568801829 podStartE2EDuration="1m7.392872653s" totalImagesPullingTime="1.824070824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:21.392313926 +0000 UTC m=+113.252393998" watchObservedRunningTime="2026-07-07 17:09:21.392872653 +0000 UTC m=+113.252952726" Jul 07 17:09:21.415273 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:21.415248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-446rp"] Jul 07 17:09:21.417625 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:21.417601 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f4fece_813b_4fff_aa86_06d7c1ca0575.slice/crio-c5edd231b9b085e8d24c1288b6b71763cc84267e13a8867301d17d56f9bf7b6f WatchSource:0}: Error finding container c5edd231b9b085e8d24c1288b6b71763cc84267e13a8867301d17d56f9bf7b6f: Status 404 returned error can't find the container with id c5edd231b9b085e8d24c1288b6b71763cc84267e13a8867301d17d56f9bf7b6f Jul 07 17:09:22.093721 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.093686 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl"] Jul 07 17:09:22.099866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.099845 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:22.103313 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.103291 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mls8q\"" Jul 07 17:09:22.105047 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.105017 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 17:09:22.105865 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.105846 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl"] Jul 07 17:09:22.196634 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.196596 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ac5ac622-54ce-40b0-9d0b-10dcbf7f6631-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-xndkl\" (UID: \"ac5ac622-54ce-40b0-9d0b-10dcbf7f6631\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:22.298152 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.298054 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ac5ac622-54ce-40b0-9d0b-10dcbf7f6631-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-xndkl\" (UID: \"ac5ac622-54ce-40b0-9d0b-10dcbf7f6631\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:22.301249 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.301187 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ac5ac622-54ce-40b0-9d0b-10dcbf7f6631-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-xndkl\" (UID: \"ac5ac622-54ce-40b0-9d0b-10dcbf7f6631\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:22.380661 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.380620 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" event={"ID":"70f4fece-813b-4fff-aa86-06d7c1ca0575","Type":"ContainerStarted","Data":"c5edd231b9b085e8d24c1288b6b71763cc84267e13a8867301d17d56f9bf7b6f"} Jul 07 17:09:22.410221 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.410129 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:22.969720 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:22.969690 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl"] Jul 07 17:09:22.973674 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:22.973607 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5ac622_54ce_40b0_9d0b_10dcbf7f6631.slice/crio-d638fb4e24682e4b767f3292cdbf37a3c1d0e2dff5733b111267969f785d1a13 WatchSource:0}: Error finding container d638fb4e24682e4b767f3292cdbf37a3c1d0e2dff5733b111267969f785d1a13: Status 404 returned error can't find the container with id d638fb4e24682e4b767f3292cdbf37a3c1d0e2dff5733b111267969f785d1a13 Jul 07 17:09:23.386507 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:23.386471 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" event={"ID":"70f4fece-813b-4fff-aa86-06d7c1ca0575","Type":"ContainerStarted","Data":"b2c0f370906151e3e83faa08290638d7bc458d3203bcfa7d5dd3229807a34036"} Jul 07 17:09:23.387898 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:23.387863 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" event={"ID":"ac5ac622-54ce-40b0-9d0b-10dcbf7f6631","Type":"ContainerStarted","Data":"d638fb4e24682e4b767f3292cdbf37a3c1d0e2dff5733b111267969f785d1a13"} Jul 07 17:09:23.406082 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:23.406028 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-446rp" podStartSLOduration=1.813432435 podStartE2EDuration="3.406009343s" totalImagesPullingTime="1.592576908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:23.403981517 +0000 UTC m=+115.264061589" watchObservedRunningTime="2026-07-07 17:09:23.406009343 +0000 UTC m=+115.266089417" Jul 07 17:09:26.403068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.403025 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" event={"ID":"b55b54c6-1aec-4c5b-ab0c-50867b607552","Type":"ContainerStarted","Data":"8493e1a3497f52f0223fa2f1234f263a24f73b17205d8b0d4a9efac7cedea67d"} Jul 07 17:09:26.403513 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.403131 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:26.404686 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.404663 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" Jul 07 17:09:26.404811 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.404755 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" event={"ID":"ac5ac622-54ce-40b0-9d0b-10dcbf7f6631","Type":"ContainerStarted","Data":"80d6ebe4e646e44cc79f8bc76c8229e06f18aeb773ae89e4007be43b7cb45a32"} Jul 07 17:09:26.404952 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.404936 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:26.409234 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.409213 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" Jul 07 17:09:26.422136 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.422097 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d7b667b7-rnhq5" podStartSLOduration=1.8418163889999999 podStartE2EDuration="6.42208513s" totalImagesPullingTime="4.580268741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:26.421233386 +0000 UTC m=+118.281313468" watchObservedRunningTime="2026-07-07 17:09:26.42208513 +0000 UTC m=+118.282165239" Jul 07 17:09:26.438097 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:26.438058 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-xndkl" podStartSLOduration=1.552232504 podStartE2EDuration="4.438042898s" totalImagesPullingTime="2.885810394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:26.437844901 +0000 UTC m=+118.297924976" watchObservedRunningTime="2026-07-07 17:09:26.438042898 +0000 UTC m=+118.298122972" Jul 07 17:09:29.928524 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:29.928492 2582 scope.go:117] "RemoveContainer" containerID="35444812c57cb1f4c6bac8a146e6922913bf1864b47a664b5a015c66844095d0" Jul 07 17:09:30.419185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.419152 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:09:30.419357 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.419266 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" event={"ID":"346488bd-0761-4f1f-a947-c476615e2deb","Type":"ContainerStarted","Data":"06362daf48fc178c44c31e1649dd644bf832ffa4f9f686ecd5c4f0adf2dac199"} Jul 07 17:09:30.419592 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.419557 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:09:30.437868 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.437819 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" podStartSLOduration=40.964914712 podStartE2EDuration="43.437805346s" totalImagesPullingTime="2.472890634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:30.437239341 +0000 UTC m=+122.297319414" watchObservedRunningTime="2026-07-07 17:09:30.437805346 +0000 UTC m=+122.297885417" Jul 07 17:09:30.697736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.697659 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-frwl4" Jul 07 17:09:30.882366 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.882330 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-8fdt6"] Jul 07 17:09:30.885465 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.885448 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:30.888201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.888172 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4bpc2\"" Jul 07 17:09:30.888201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.888189 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 17:09:30.888370 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.888226 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 17:09:30.903081 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.903052 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-8fdt6"] Jul 07 17:09:30.968416 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:30.968339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxclg\" (UniqueName: \"kubernetes.io/projected/45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9-kube-api-access-hxclg\") pod \"downloads-685bfb5f64-8fdt6\" (UID: \"45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9\") " pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:31.069596 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.069543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxclg\" (UniqueName: \"kubernetes.io/projected/45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9-kube-api-access-hxclg\") pod \"downloads-685bfb5f64-8fdt6\" (UID: \"45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9\") " pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:31.080508 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.080470 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxclg\" (UniqueName: \"kubernetes.io/projected/45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9-kube-api-access-hxclg\") pod \"downloads-685bfb5f64-8fdt6\" (UID: \"45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9\") " pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:31.194587 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.194547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:31.334540 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.334505 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-8fdt6"] Jul 07 17:09:31.337848 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:31.337823 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d4ecd5_0df8_4ea0_8cfd_20fb7fa2ebf9.slice/crio-6ccb4a129d770ce07748df53902758ab60fb18205b7f9f568632e24881e9988b WatchSource:0}: Error finding container 6ccb4a129d770ce07748df53902758ab60fb18205b7f9f568632e24881e9988b: Status 404 returned error can't find the container with id 6ccb4a129d770ce07748df53902758ab60fb18205b7f9f568632e24881e9988b Jul 07 17:09:31.382865 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.382845 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4jnln" Jul 07 17:09:31.426231 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.426198 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-8fdt6" event={"ID":"45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9","Type":"ContainerStarted","Data":"6ccb4a129d770ce07748df53902758ab60fb18205b7f9f568632e24881e9988b"} Jul 07 17:09:31.549455 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.549373 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk"] Jul 07 17:09:31.553892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.553868 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.556722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.556697 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 17:09:31.556852 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.556786 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 17:09:31.556852 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.556817 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 17:09:31.556953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.556931 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8xvn7\"" Jul 07 17:09:31.557169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.557152 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 17:09:31.557263 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.557248 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 17:09:31.570586 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.570551 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk"] Jul 07 17:09:31.571456 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.571435 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj"] Jul 07 17:09:31.576902 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.576235 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.579928 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.579910 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bj5pl\"" Jul 07 17:09:31.580078 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.580057 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 17:09:31.580146 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.580116 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 17:09:31.580212 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.580121 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 17:09:31.582602 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.582582 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gcznj"] Jul 07 17:09:31.586334 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.586255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.588516 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.588498 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj"] Jul 07 17:09:31.588700 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.588679 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 17:09:31.588700 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.588697 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vvz5v\"" Jul 07 17:09:31.588845 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.588823 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 17:09:31.588938 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.588919 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 17:09:31.678493 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678493 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678491 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cp2s\" (UniqueName: \"kubernetes.io/projected/8e7a1707-3649-415f-9049-742731e1172a-kube-api-access-8cp2s\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.678722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678510 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fd6-2578-446b-a21f-61ed365b16c7-metrics-client-ca\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwjvh\" (UniqueName: \"kubernetes.io/projected/8a1d7fd6-2578-446b-a21f-61ed365b16c7-kube-api-access-rwjvh\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678561 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.678722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678662 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.678722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.678901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678731 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.678901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-sys\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678766 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-root\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.678901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-tls\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.678901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.679120 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678906 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-wtmp\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.679120 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.679120 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr5g2\" (UniqueName: \"kubernetes.io/projected/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-api-access-dr5g2\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.679120 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.678990 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e7a1707-3649-415f-9049-742731e1172a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.679120 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.679008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-textfile\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.779880 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779850 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.779880 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-sys\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779906 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-root\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779935 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-tls\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779968 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-sys\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.779980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780021 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-wtmp\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780035 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-root\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780080 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780096 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr5g2\" (UniqueName: \"kubernetes.io/projected/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-api-access-dr5g2\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e7a1707-3649-415f-9049-742731e1172a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780162 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-textfile\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780185 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-wtmp\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780192 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780248 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cp2s\" (UniqueName: \"kubernetes.io/projected/8e7a1707-3649-415f-9049-742731e1172a-kube-api-access-8cp2s\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fd6-2578-446b-a21f-61ed365b16c7-metrics-client-ca\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwjvh\" (UniqueName: \"kubernetes.io/projected/8a1d7fd6-2578-446b-a21f-61ed365b16c7-kube-api-access-rwjvh\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.780451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780420 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780757 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.780767 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.781193 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-textfile\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:09:31.781212 2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.781226 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:09:31.781283 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-tls podName:8e7a1707-3649-415f-9049-742731e1172a nodeName:}" failed. No retries permitted until 2026-07-07 17:09:32.28125566 +0000 UTC m=+124.141335719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-8fxtk" (UID: "8e7a1707-3649-415f-9049-742731e1172a") : secret "openshift-state-metrics-tls" not found Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.781400 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fd6-2578-446b-a21f-61ed365b16c7-metrics-client-ca\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.781491 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.781793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.781511 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e7a1707-3649-415f-9049-742731e1172a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.782727 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.782696 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-tls\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.783892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.783592 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.783892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.783844 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a1d7fd6-2578-446b-a21f-61ed365b16c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.784588 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.784529 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.784759 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.784618 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.796918 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.796883 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwjvh\" (UniqueName: \"kubernetes.io/projected/8a1d7fd6-2578-446b-a21f-61ed365b16c7-kube-api-access-rwjvh\") pod \"node-exporter-gcznj\" (UID: \"8a1d7fd6-2578-446b-a21f-61ed365b16c7\") " pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.798285 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.798264 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr5g2\" (UniqueName: \"kubernetes.io/projected/c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a-kube-api-access-dr5g2\") pod \"kube-state-metrics-7c9dbb8b7d-jxddj\" (UID: \"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.798458 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.798438 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cp2s\" (UniqueName: \"kubernetes.io/projected/8e7a1707-3649-415f-9049-742731e1172a-kube-api-access-8cp2s\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:31.885564 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.885529 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" Jul 07 17:09:31.894779 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:31.894756 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gcznj" Jul 07 17:09:31.904433 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:31.904406 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1d7fd6_2578_446b_a21f_61ed365b16c7.slice/crio-7f33e40759ba3bca581d94e18a5f595feb01e192098ff16b8956164267187d6e WatchSource:0}: Error finding container 7f33e40759ba3bca581d94e18a5f595feb01e192098ff16b8956164267187d6e: Status 404 returned error can't find the container with id 7f33e40759ba3bca581d94e18a5f595feb01e192098ff16b8956164267187d6e Jul 07 17:09:32.015868 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.015843 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj"] Jul 07 17:09:32.018189 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:32.018158 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73e2d5b_9f13_4165_bb4b_c4a6f2d1062a.slice/crio-1940b7418fbbd23358c7144fec0b8c991d6e636af90653fbf9bec4b4d24ec075 WatchSource:0}: Error finding container 1940b7418fbbd23358c7144fec0b8c991d6e636af90653fbf9bec4b4d24ec075: Status 404 returned error can't find the container with id 1940b7418fbbd23358c7144fec0b8c991d6e636af90653fbf9bec4b4d24ec075 Jul 07 17:09:32.284427 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.284341 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:32.286715 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.286693 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a1707-3649-415f-9049-742731e1172a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8fxtk\" (UID: \"8e7a1707-3649-415f-9049-742731e1172a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:32.431621 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.431560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gcznj" event={"ID":"8a1d7fd6-2578-446b-a21f-61ed365b16c7","Type":"ContainerStarted","Data":"7f33e40759ba3bca581d94e18a5f595feb01e192098ff16b8956164267187d6e"} Jul 07 17:09:32.433245 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.433201 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" event={"ID":"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a","Type":"ContainerStarted","Data":"1940b7418fbbd23358c7144fec0b8c991d6e636af90653fbf9bec4b4d24ec075"} Jul 07 17:09:32.463570 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.463534 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" Jul 07 17:09:32.624558 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:32.624528 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk"] Jul 07 17:09:32.627833 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:32.627803 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7a1707_3649_415f_9049_742731e1172a.slice/crio-1fba5f6cdf875a64190ce5e88d2741e5b64d9a585f8d0c061dc2328eb918250c WatchSource:0}: Error finding container 1fba5f6cdf875a64190ce5e88d2741e5b64d9a585f8d0c061dc2328eb918250c: Status 404 returned error can't find the container with id 1fba5f6cdf875a64190ce5e88d2741e5b64d9a585f8d0c061dc2328eb918250c Jul 07 17:09:33.441239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:33.441040 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" event={"ID":"8e7a1707-3649-415f-9049-742731e1172a","Type":"ContainerStarted","Data":"38d0ea8accc70c02c869daa5915ecbd15634793e832d344a26818bf104afeb6e"} Jul 07 17:09:33.441239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:33.441121 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" event={"ID":"8e7a1707-3649-415f-9049-742731e1172a","Type":"ContainerStarted","Data":"37f4ce7f0967ccccd942556660302a0874f2df3bfd8d99eac705a6e87685da10"} Jul 07 17:09:33.441239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:33.441152 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" event={"ID":"8e7a1707-3649-415f-9049-742731e1172a","Type":"ContainerStarted","Data":"1fba5f6cdf875a64190ce5e88d2741e5b64d9a585f8d0c061dc2328eb918250c"} Jul 07 17:09:33.443623 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:33.443560 2582 generic.go:356] "Generic (PLEG): container finished" podID="8a1d7fd6-2578-446b-a21f-61ed365b16c7" containerID="acefa0c90979772fdc1af0ccff4219a904615478ead17ae8e7a81a3f99cd03d8" exitCode=0 Jul 07 17:09:33.443623 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:33.443626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gcznj" event={"ID":"8a1d7fd6-2578-446b-a21f-61ed365b16c7","Type":"ContainerDied","Data":"acefa0c90979772fdc1af0ccff4219a904615478ead17ae8e7a81a3f99cd03d8"} Jul 07 17:09:34.451511 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.451467 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" event={"ID":"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a","Type":"ContainerStarted","Data":"4972ff4cea9a299bdb347f8e7a02ccc80218cc63677d267ebc2c9633e5715f28"} Jul 07 17:09:34.451926 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.451602 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" event={"ID":"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a","Type":"ContainerStarted","Data":"cbd03c35d1fc8bc2195c3b6dbb850e804f2504b3e11d2d6fcde70f3d24647cfc"} Jul 07 17:09:34.457774 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.457629 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" event={"ID":"8e7a1707-3649-415f-9049-742731e1172a","Type":"ContainerStarted","Data":"fc4a4f70b3119b9f7f733fb987db8e79445fef046b5b87fd69b68586c1a68a60"} Jul 07 17:09:34.464205 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.464138 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gcznj" event={"ID":"8a1d7fd6-2578-446b-a21f-61ed365b16c7","Type":"ContainerStarted","Data":"5de5bbd0b299cb1896f6b248d6090ef5020743d4ff05e361ebccc21e216dfbaf"} Jul 07 17:09:34.464350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.464222 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gcznj" event={"ID":"8a1d7fd6-2578-446b-a21f-61ed365b16c7","Type":"ContainerStarted","Data":"5c14750b4ad465bd746385b762d561b3c817fc85b15b69a7cfa917790db19d69"} Jul 07 17:09:34.493900 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.493796 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8fxtk" podStartSLOduration=1.9958425499999999 podStartE2EDuration="3.493773572s" totalImagesPullingTime="1.497931022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:34.492558946 +0000 UTC m=+126.352639019" watchObservedRunningTime="2026-07-07 17:09:34.493773572 +0000 UTC m=+126.353853627" Jul 07 17:09:34.527071 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.526486 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gcznj" podStartSLOduration=2.24136093 podStartE2EDuration="3.526462449s" totalImagesPullingTime="1.210258298s" totalInitContainerRuntime="74.843221ms" isStatefulPod=true podCreationTimestamp="2026-07-07 17:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:34.525324917 +0000 UTC m=+126.385404991" watchObservedRunningTime="2026-07-07 17:09:34.526462449 +0000 UTC m=+126.386542522" Jul 07 17:09:34.615317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.615285 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b79df7448-nbddj"] Jul 07 17:09:34.620323 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.620255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.624310 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.623893 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 17:09:34.624310 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.624250 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 17:09:34.624310 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.624258 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 17:09:34.624664 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.624613 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 17:09:34.624965 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.624819 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d34uligsmik4m\"" Jul 07 17:09:34.624965 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.624863 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 17:09:34.636912 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.636894 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q8tvl\"" Jul 07 17:09:34.709903 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.709863 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710173 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.709914 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710173 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.709954 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710173 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.710009 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-tls\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.710181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.710212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-grpc-tls\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.710262 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-metrics-client-ca\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.710368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.710356 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb5xs\" (UniqueName: \"kubernetes.io/projected/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-kube-api-access-jb5xs\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.712557 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.712532 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b79df7448-nbddj"] Jul 07 17:09:34.811419 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-tls\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811419 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811419 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-grpc-tls\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811433 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-metrics-client-ca\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb5xs\" (UniqueName: \"kubernetes.io/projected/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-kube-api-access-jb5xs\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811621 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.811723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.811659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.812425 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.812365 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-metrics-client-ca\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.814435 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.814386 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-tls\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.814661 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.814609 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.814769 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.814711 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.814919 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.814893 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.815030 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.815014 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.815378 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.815357 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-secret-grpc-tls\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.824190 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.824159 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb5xs\" (UniqueName: \"kubernetes.io/projected/ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1-kube-api-access-jb5xs\") pod \"thanos-querier-7b79df7448-nbddj\" (UID: \"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1\") " pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:34.932213 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:34.932173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:35.094903 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.094880 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b79df7448-nbddj"] Jul 07 17:09:35.096734 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:35.096699 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6a3c0c_7b17_4f8a_bf9c_7fed19944ee1.slice/crio-529b97f385621f96c202c693b5ff4b2fe70a06e71f6c1de803db007ea17ad82a WatchSource:0}: Error finding container 529b97f385621f96c202c693b5ff4b2fe70a06e71f6c1de803db007ea17ad82a: Status 404 returned error can't find the container with id 529b97f385621f96c202c693b5ff4b2fe70a06e71f6c1de803db007ea17ad82a Jul 07 17:09:35.469515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.469482 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"529b97f385621f96c202c693b5ff4b2fe70a06e71f6c1de803db007ea17ad82a"} Jul 07 17:09:35.471530 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.471493 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" event={"ID":"c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a","Type":"ContainerStarted","Data":"c3113dae8ce209ee6fea56d78bf4a3abe023c8a46aa9708413419f9a729b3100"} Jul 07 17:09:35.493312 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.493257 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jxddj" podStartSLOduration=2.238298269 podStartE2EDuration="4.493239228s" totalImagesPullingTime="2.254940959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:35.492447229 +0000 UTC m=+127.352527303" watchObservedRunningTime="2026-07-07 17:09:35.493239228 +0000 UTC m=+127.353319299" Jul 07 17:09:35.965065 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.965028 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-66b546c5b5-nsc5n"] Jul 07 17:09:35.968505 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.968479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:35.989670 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.989648 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 17:09:35.990697 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.990667 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 17:09:35.990867 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.990850 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 17:09:35.992502 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.992479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 17:09:35.992636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.992512 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-g7cst\"" Jul 07 17:09:35.992636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:35.992561 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-18hvl5hg9dif1\"" Jul 07 17:09:36.006346 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.005244 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66b546c5b5-nsc5n"] Jul 07 17:09:36.023840 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.023808 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpmx9\" (UniqueName: \"kubernetes.io/projected/9c80680a-17b4-40fa-b64c-b324dcb57341-kube-api-access-rpmx9\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.023989 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.023869 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c80680a-17b4-40fa-b64c-b324dcb57341-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.023989 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.023945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-secret-metrics-server-tls\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.024119 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.023991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-client-ca-bundle\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.024119 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.024016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9c80680a-17b4-40fa-b64c-b324dcb57341-metrics-server-audit-profiles\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.024119 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.024043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9c80680a-17b4-40fa-b64c-b324dcb57341-audit-log\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.024280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.024117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-secret-metrics-server-client-certs\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125249 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125212 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpmx9\" (UniqueName: \"kubernetes.io/projected/9c80680a-17b4-40fa-b64c-b324dcb57341-kube-api-access-rpmx9\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c80680a-17b4-40fa-b64c-b324dcb57341-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-secret-metrics-server-tls\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-client-ca-bundle\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125374 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9c80680a-17b4-40fa-b64c-b324dcb57341-metrics-server-audit-profiles\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125400 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9c80680a-17b4-40fa-b64c-b324dcb57341-audit-log\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.125428 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.125430 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-secret-metrics-server-client-certs\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.126162 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.126104 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9c80680a-17b4-40fa-b64c-b324dcb57341-audit-log\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.126639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.126606 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c80680a-17b4-40fa-b64c-b324dcb57341-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.126891 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.126857 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9c80680a-17b4-40fa-b64c-b324dcb57341-metrics-server-audit-profiles\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.128416 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.128391 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-secret-metrics-server-client-certs\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.128521 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.128498 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-client-ca-bundle\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.129022 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.129003 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9c80680a-17b4-40fa-b64c-b324dcb57341-secret-metrics-server-tls\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.135162 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.135134 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpmx9\" (UniqueName: \"kubernetes.io/projected/9c80680a-17b4-40fa-b64c-b324dcb57341-kube-api-access-rpmx9\") pod \"metrics-server-66b546c5b5-nsc5n\" (UID: \"9c80680a-17b4-40fa-b64c-b324dcb57341\") " pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.279696 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.279606 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:36.438865 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.438834 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66b546c5b5-nsc5n"] Jul 07 17:09:36.441886 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:36.441851 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c80680a_17b4_40fa_b64c_b324dcb57341.slice/crio-04673a90c06988ed4dd93ae6becf87baf76d5c6e75ae2dea1f55a90f08fc06c8 WatchSource:0}: Error finding container 04673a90c06988ed4dd93ae6becf87baf76d5c6e75ae2dea1f55a90f08fc06c8: Status 404 returned error can't find the container with id 04673a90c06988ed4dd93ae6becf87baf76d5c6e75ae2dea1f55a90f08fc06c8 Jul 07 17:09:36.478239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.478197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" event={"ID":"9c80680a-17b4-40fa-b64c-b324dcb57341","Type":"ContainerStarted","Data":"04673a90c06988ed4dd93ae6becf87baf76d5c6e75ae2dea1f55a90f08fc06c8"} Jul 07 17:09:36.788391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.788351 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-688bfdc6b7-726h2"] Jul 07 17:09:36.793551 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.793527 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.796470 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.796261 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 17:09:36.796637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.796305 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 17:09:36.796708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.796318 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 17:09:36.796798 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.796358 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2qm5n\"" Jul 07 17:09:36.796889 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.796406 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 17:09:36.797075 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.796450 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 17:09:36.802474 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.802450 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 17:09:36.806679 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.806658 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-688bfdc6b7-726h2"] Jul 07 17:09:36.932606 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932554 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-telemeter-client-tls\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932797 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932664 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-metrics-client-ca\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932797 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-federate-client-tls\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932797 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932765 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-serving-certs-ca-bundle\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932831 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932871 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgj4q\" (UniqueName: \"kubernetes.io/projected/4e23bade-2f11-45f0-993f-8fc9e903fd7d-kube-api-access-cgj4q\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932904 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:36.932993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:36.932970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-secret-telemeter-client\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034330 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034528 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-secret-telemeter-client\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034640 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034542 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-telemeter-client-tls\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034698 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-metrics-client-ca\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034698 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-federate-client-tls\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-serving-certs-ca-bundle\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034777 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.034894 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.034821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgj4q\" (UniqueName: \"kubernetes.io/projected/4e23bade-2f11-45f0-993f-8fc9e903fd7d-kube-api-access-cgj4q\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.035426 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.035348 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.035951 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.035926 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-metrics-client-ca\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.036064 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.035927 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e23bade-2f11-45f0-993f-8fc9e903fd7d-serving-certs-ca-bundle\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.037701 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.037652 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-secret-telemeter-client\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.037866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.037844 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-telemeter-client-tls\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.037940 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.037882 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-federate-client-tls\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.038224 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.038197 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e23bade-2f11-45f0-993f-8fc9e903fd7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.045769 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.045708 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgj4q\" (UniqueName: \"kubernetes.io/projected/4e23bade-2f11-45f0-993f-8fc9e903fd7d-kube-api-access-cgj4q\") pod \"telemeter-client-688bfdc6b7-726h2\" (UID: \"4e23bade-2f11-45f0-993f-8fc9e903fd7d\") " pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.106794 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.106760 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" Jul 07 17:09:37.263211 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.263175 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-688bfdc6b7-726h2"] Jul 07 17:09:37.266254 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:37.266218 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e23bade_2f11_45f0_993f_8fc9e903fd7d.slice/crio-f03d2414d476f71d01330486422c9d13580b1df8f07179a29a4dd5d2e0cca53b WatchSource:0}: Error finding container f03d2414d476f71d01330486422c9d13580b1df8f07179a29a4dd5d2e0cca53b: Status 404 returned error can't find the container with id f03d2414d476f71d01330486422c9d13580b1df8f07179a29a4dd5d2e0cca53b Jul 07 17:09:37.486055 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.485944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" event={"ID":"4e23bade-2f11-45f0-993f-8fc9e903fd7d","Type":"ContainerStarted","Data":"f03d2414d476f71d01330486422c9d13580b1df8f07179a29a4dd5d2e0cca53b"} Jul 07 17:09:37.973887 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.973701 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:09:37.979975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.979944 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:37.983341 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.983143 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 17:09:37.983341 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.983227 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 17:09:37.983747 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.983720 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 17:09:37.984016 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.983995 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 17:09:37.985594 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.985392 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kx5xw\"" Jul 07 17:09:37.986731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.985841 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 17:09:37.986731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.986070 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 17:09:37.986731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.986649 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 17:09:37.988244 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.988221 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 17:09:37.992140 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.991390 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 17:09:37.992140 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.991468 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 17:09:37.992140 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.991619 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 17:09:37.993993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.993899 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-donmeildpvgpf\"" Jul 07 17:09:37.994784 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.994764 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 17:09:37.999075 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:37.999055 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 17:09:38.001938 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.001911 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 17:09:38.004813 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.004793 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:09:38.045539 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045499 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-config-out\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045544 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045676 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-web-config\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045703 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045739 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-config\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.045976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.045923 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046032 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046082 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046132 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxj4b\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-kube-api-access-vxj4b\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046172 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046223 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046269 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046294 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046326 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.046389 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.046363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147396 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147488 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147535 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147605 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxj4b\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-kube-api-access-vxj4b\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147758 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147828 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-config-out\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-web-config\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.148368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.147933 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148171 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148293 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148703 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148860 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-config\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.148881 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.149327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.149223 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.150376 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.150346 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.151161 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.150855 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.152035 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.151678 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.153532 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.153509 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.155131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.154744 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-config\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.155131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.155009 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.155131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.155097 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-web-config\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.156233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.155815 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.156233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.155989 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.156233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.156193 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.156233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.156222 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-config-out\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.156484 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.156270 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.157313 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.157288 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.158976 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.158952 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.159087 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.159000 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxj4b\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-kube-api-access-vxj4b\") pod \"prometheus-k8s-0\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.300639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.300612 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:38.497517 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.497311 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"617694292e5e356b299baaab093cdd57864821b7f015e9684fb7a03a45d9a8b7"} Jul 07 17:09:38.497517 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.497392 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"48955481e723fc30141831d990922ebb53982082039b84152f4bfc6a11f6a16a"} Jul 07 17:09:38.497517 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.497427 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"d823262030c3f8977f4cd06f3b46d8a2381f28c4156dc0deb3437efe26d52736"} Jul 07 17:09:38.514320 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.514215 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:09:38.527620 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:38.524226 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eae52ac_f711_47a1_b7c7_2c324a856890.slice/crio-d21e381be33a0bb3c972f8faf1cc98cf0fb4b448b7e64d59432e9907bd17a695 WatchSource:0}: Error finding container d21e381be33a0bb3c972f8faf1cc98cf0fb4b448b7e64d59432e9907bd17a695: Status 404 returned error can't find the container with id d21e381be33a0bb3c972f8faf1cc98cf0fb4b448b7e64d59432e9907bd17a695 Jul 07 17:09:38.552523 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.552493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:09:38.556037 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.556002 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c3d23-9652-4f2b-acee-d535a4f662f6-metrics-certs\") pod \"network-metrics-daemon-7n5g9\" (UID: \"158c3d23-9652-4f2b-acee-d535a4f662f6\") " pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:09:38.651670 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.651630 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-chtgz\"" Jul 07 17:09:38.659286 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:38.659259 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7n5g9" Jul 07 17:09:39.182020 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:39.181994 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7n5g9"] Jul 07 17:09:39.184396 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:09:39.184370 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158c3d23_9652_4f2b_acee_d535a4f662f6.slice/crio-79703dba44ce38c0a86226a52aae97238b386e4987072cfa87cdfcec09a0ece2 WatchSource:0}: Error finding container 79703dba44ce38c0a86226a52aae97238b386e4987072cfa87cdfcec09a0ece2: Status 404 returned error can't find the container with id 79703dba44ce38c0a86226a52aae97238b386e4987072cfa87cdfcec09a0ece2 Jul 07 17:09:39.515681 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:39.515538 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7n5g9" event={"ID":"158c3d23-9652-4f2b-acee-d535a4f662f6","Type":"ContainerStarted","Data":"79703dba44ce38c0a86226a52aae97238b386e4987072cfa87cdfcec09a0ece2"} Jul 07 17:09:39.517631 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:39.517601 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"d21e381be33a0bb3c972f8faf1cc98cf0fb4b448b7e64d59432e9907bd17a695"} Jul 07 17:09:39.520552 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:39.520520 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" event={"ID":"9c80680a-17b4-40fa-b64c-b324dcb57341","Type":"ContainerStarted","Data":"0d0e88b1810153a067ddca209c20f7b2418f89f8cad4c837e08f5a2bb5ec2a83"} Jul 07 17:09:39.541700 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:39.541636 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" podStartSLOduration=1.8738879929999999 podStartE2EDuration="4.541616075s" totalImagesPullingTime="2.667728082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:39.540226064 +0000 UTC m=+131.400306161" watchObservedRunningTime="2026-07-07 17:09:39.541616075 +0000 UTC m=+131.401696148" Jul 07 17:09:40.350449 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:40.350078 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:09:40.537842 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:40.537640 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"8520173eafe31761deef9ad3c44ef4cdfa64fe615ec072b3c762ab02a0d33547"} Jul 07 17:09:41.552184 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.552027 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"3a8cd7e57e297cf5e210ba2f6cdc2842c43d2cf85e7c232daea0c92be712f756"} Jul 07 17:09:41.552184 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.552118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" event={"ID":"ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1","Type":"ContainerStarted","Data":"e0df20dfda3ca7aa7ca4c5907628d1f3edd73ecef06dbb0e05cd0f503c351078"} Jul 07 17:09:41.553161 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.553134 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:41.554854 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.554833 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7n5g9" event={"ID":"158c3d23-9652-4f2b-acee-d535a4f662f6","Type":"ContainerStarted","Data":"04d1ec9913511aa12b2cbd2b4e1914a03f63269a8a625800d25863e6356ec53e"} Jul 07 17:09:41.556984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.556951 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" exitCode=0 Jul 07 17:09:41.557093 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.557080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} Jul 07 17:09:41.560617 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.560278 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" event={"ID":"4e23bade-2f11-45f0-993f-8fc9e903fd7d","Type":"ContainerStarted","Data":"ae7e0bcd275b933259689a70668c93cb8a9a27bfb94f4e6289b707459ef155b3"} Jul 07 17:09:41.560617 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.560327 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" event={"ID":"4e23bade-2f11-45f0-993f-8fc9e903fd7d","Type":"ContainerStarted","Data":"f7fed25f2d6d68e3e2b2c17e759da7adec8beb1f1c5d98667cf3ae006e26df03"} Jul 07 17:09:41.560617 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.560355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" event={"ID":"4e23bade-2f11-45f0-993f-8fc9e903fd7d","Type":"ContainerStarted","Data":"5b0b269d4a6acb0d0b5b80803ef232ffa72dd2730bf5e17de3ec61207ffc5f88"} Jul 07 17:09:41.585696 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.582226 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" podStartSLOduration=3.318123559 podStartE2EDuration="7.582208314s" totalImagesPullingTime="4.264084755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:41.57747177 +0000 UTC m=+133.437551843" watchObservedRunningTime="2026-07-07 17:09:41.582208314 +0000 UTC m=+133.442288388" Jul 07 17:09:41.634280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:41.634228 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-688bfdc6b7-726h2" podStartSLOduration=2.453617211 podStartE2EDuration="5.634212459s" totalImagesPullingTime="3.180595248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:41.632646356 +0000 UTC m=+133.492726428" watchObservedRunningTime="2026-07-07 17:09:41.634212459 +0000 UTC m=+133.494292531" Jul 07 17:09:42.566767 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:42.566706 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7n5g9" event={"ID":"158c3d23-9652-4f2b-acee-d535a4f662f6","Type":"ContainerStarted","Data":"3234d9a54ad1720c49c92bece84f31d22c9bfe79fd19374f1f8a7a6d40aa683e"} Jul 07 17:09:42.586495 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:42.586435 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7n5g9" podStartSLOduration=131.385939484 podStartE2EDuration="2m13.586418566s" totalImagesPullingTime="2.200479082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:42.585812413 +0000 UTC m=+134.445892485" watchObservedRunningTime="2026-07-07 17:09:42.586418566 +0000 UTC m=+134.446498637" Jul 07 17:09:43.568161 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:43.564151 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67447d8bcd-fqdlm"] Jul 07 17:09:43.586656 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:43.586625 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b79df7448-nbddj" Jul 07 17:09:46.597378 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:46.597345 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} Jul 07 17:09:46.597821 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:46.597431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} Jul 07 17:09:47.607107 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:47.607048 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} Jul 07 17:09:47.607567 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:47.607199 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} Jul 07 17:09:47.607567 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:47.607241 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} Jul 07 17:09:47.607567 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:47.607274 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerStarted","Data":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} Jul 07 17:09:47.641239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:47.641178 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.062707787 podStartE2EDuration="10.641161681s" totalImagesPullingTime="6.548614116s" totalInitContainerRuntime="29.839778ms" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:47.637880475 +0000 UTC m=+139.497960547" watchObservedRunningTime="2026-07-07 17:09:47.641161681 +0000 UTC m=+139.501241753" Jul 07 17:09:48.300846 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:48.300813 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:09:56.280098 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:56.280009 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:56.280566 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:56.280102 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:09:56.651555 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:56.651496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-8fdt6" event={"ID":"45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9","Type":"ContainerStarted","Data":"3a86a5dcdb79ced4ddb876571c5ce119e9828a71ea1b3abf8efca3f8270db62c"} Jul 07 17:09:56.651771 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:56.651689 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:56.668232 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:56.668205 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-8fdt6" Jul 07 17:09:56.674187 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:56.674130 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-8fdt6" podStartSLOduration=2.121662701 podStartE2EDuration="26.674110956s" totalImagesPullingTime="24.552448255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:56.671492066 +0000 UTC m=+148.531572138" watchObservedRunningTime="2026-07-07 17:09:56.674110956 +0000 UTC m=+148.534191028" Jul 07 17:09:57.657916 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:57.657882 2582 generic.go:356] "Generic (PLEG): container finished" podID="d983025a-e880-4d95-a161-da213fda040b" containerID="a35f247a1d07754de08d37dc729685a03d9ca294ffdd330d1f48ce4ea1138936" exitCode=0 Jul 07 17:09:57.658402 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:57.657970 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" event={"ID":"d983025a-e880-4d95-a161-da213fda040b","Type":"ContainerDied","Data":"a35f247a1d07754de08d37dc729685a03d9ca294ffdd330d1f48ce4ea1138936"} Jul 07 17:09:57.658516 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:57.658495 2582 scope.go:117] "RemoveContainer" containerID="a35f247a1d07754de08d37dc729685a03d9ca294ffdd330d1f48ce4ea1138936" Jul 07 17:09:58.672340 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:09:58.672294 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6x5tf" event={"ID":"d983025a-e880-4d95-a161-da213fda040b","Type":"ContainerStarted","Data":"0bca169a82129869c21c0a1d0b164b46a18dec6267620e2c5a44931fe6cc0374"} Jul 07 17:10:08.595422 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.595374 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" podUID="e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" containerName="registry" containerID="cri-o://b3ce3ea9c7f9802bd91b22361baeaa1e2444e48519303a64423a68d92f6f489b" gracePeriod=30 Jul 07 17:10:08.720656 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.720621 2582 generic.go:356] "Generic (PLEG): container finished" podID="e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" containerID="b3ce3ea9c7f9802bd91b22361baeaa1e2444e48519303a64423a68d92f6f489b" exitCode=0 Jul 07 17:10:08.720821 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.720695 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" event={"ID":"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb","Type":"ContainerDied","Data":"b3ce3ea9c7f9802bd91b22361baeaa1e2444e48519303a64423a68d92f6f489b"} Jul 07 17:10:08.877204 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.877172 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:10:08.942751 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.942719 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-ca-trust-extracted\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.942751 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.942752 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.942995 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.942772 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-bound-sa-token\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.942995 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.942816 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fr8w\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-kube-api-access-5fr8w\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.942995 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.942849 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-certificates\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.943160 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.943016 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-trusted-ca\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.943160 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.943103 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-installation-pull-secrets\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.943160 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.943143 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-image-registry-private-configuration\") pod \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\" (UID: \"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb\") " Jul 07 17:10:08.943641 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.943310 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:08.943641 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.943428 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:08.943641 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.943496 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-certificates\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:08.945418 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.945392 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:10:08.945676 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.945659 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:10:08.945747 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.945687 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:08.945747 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.945725 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-kube-api-access-5fr8w" (OuterVolumeSpecName: "kube-api-access-5fr8w") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "kube-api-access-5fr8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:10:08.945906 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.945886 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:08.951902 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:08.951878 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" (UID: "e59b48ea-ba2f-4eb5-b74c-1f246b8465fb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:10:09.044816 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044781 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-installation-pull-secrets\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.044816 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044812 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-image-registry-private-configuration\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.044816 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044823 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-ca-trust-extracted\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.045057 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044835 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-registry-tls\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.045057 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044845 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-bound-sa-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.045057 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044852 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fr8w\" (UniqueName: \"kubernetes.io/projected/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-kube-api-access-5fr8w\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.045057 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.044860 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb-trusted-ca\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:09.725624 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.725557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" event={"ID":"e59b48ea-ba2f-4eb5-b74c-1f246b8465fb","Type":"ContainerDied","Data":"51c763448008e6095f7c5d32186b0cce034d8d9eba4aa8894e67ac518dc9e386"} Jul 07 17:10:09.726088 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.725682 2582 scope.go:117] "RemoveContainer" containerID="b3ce3ea9c7f9802bd91b22361baeaa1e2444e48519303a64423a68d92f6f489b" Jul 07 17:10:09.726088 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.725606 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67447d8bcd-fqdlm" Jul 07 17:10:09.751153 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.751127 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67447d8bcd-fqdlm"] Jul 07 17:10:09.754980 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:09.754955 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67447d8bcd-fqdlm"] Jul 07 17:10:10.934076 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:10.934042 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" path="/var/lib/kubelet/pods/e59b48ea-ba2f-4eb5-b74c-1f246b8465fb/volumes" Jul 07 17:10:16.285187 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:16.285160 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:10:16.289185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:16.289161 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-66b546c5b5-nsc5n" Jul 07 17:10:38.300989 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:38.300949 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:38.358728 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:38.358696 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:38.860625 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:38.860598 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:56.342009 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.341976 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:10:56.342518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.342276 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="prometheus" containerID="cri-o://a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" gracePeriod=600 Jul 07 17:10:56.342518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.342318 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-thanos" containerID="cri-o://983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" gracePeriod=600 Jul 07 17:10:56.342518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.342320 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="thanos-sidecar" containerID="cri-o://df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" gracePeriod=600 Jul 07 17:10:56.342518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.342351 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="config-reloader" containerID="cri-o://29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" gracePeriod=600 Jul 07 17:10:56.342518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.342331 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-web" containerID="cri-o://8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" gracePeriod=600 Jul 07 17:10:56.342518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.342406 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy" containerID="cri-o://7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" gracePeriod=600 Jul 07 17:10:56.585245 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.585218 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:56.661553 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661525 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-trusted-ca-bundle\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.661740 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661560 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-kubelet-serving-ca-bundle\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.661740 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661599 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-grpc-tls\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.661740 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661628 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-tls\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.661907 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661769 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-config\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.661907 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661817 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.661907 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661878 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-0\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661909 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-db\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661947 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-config-out\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661973 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-2\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.661989 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662015 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-tls-assets\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662026 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.662062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662045 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-serving-certs-ca-bundle\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662070 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-1\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662108 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-web-config\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662137 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-metrics-client-ca\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662178 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-kube-rbac-proxy\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662215 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-thanos-prometheus-http-client-file\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662255 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-metrics-client-certs\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662279 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxj4b\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-kube-api-access-vxj4b\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662317 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4eae52ac-f711-47a1-b7c7-2c324a856890\" (UID: \"4eae52ac-f711-47a1-b7c7-2c324a856890\") " Jul 07 17:10:56.662793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662440 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.662793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662645 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.662793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662664 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.662793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662680 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.663004 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.662980 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.664054 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.663881 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:10:56.664180 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.664116 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.664784 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.664757 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.664928 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.664903 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:10:56.665002 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.664984 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.666262 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.666236 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.666588 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.666533 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.666689 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.666561 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:10:56.666689 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.666562 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-config-out" (OuterVolumeSpecName: "config-out") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:10:56.666813 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.666687 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-config" (OuterVolumeSpecName: "config") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.666963 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.666924 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.667081 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.667028 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-kube-api-access-vxj4b" (OuterVolumeSpecName: "kube-api-access-vxj4b") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "kube-api-access-vxj4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:10:56.667161 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.667137 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.667231 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.667209 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.668088 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.668061 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.678373 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.678348 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-web-config" (OuterVolumeSpecName: "web-config") pod "4eae52ac-f711-47a1-b7c7-2c324a856890" (UID: "4eae52ac-f711-47a1-b7c7-2c324a856890"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:10:56.763537 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763502 2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-kube-rbac-proxy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763537 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763537 2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763537 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763549 2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-metrics-client-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763559 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxj4b\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-kube-api-access-vxj4b\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763567 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763603 2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-grpc-tls\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763614 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763622 2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-config\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763631 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763640 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763648 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-db\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763657 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4eae52ac-f711-47a1-b7c7-2c324a856890-config-out\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763665 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4eae52ac-f711-47a1-b7c7-2c324a856890-tls-assets\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763673 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763681 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763689 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4eae52ac-f711-47a1-b7c7-2c324a856890-web-config\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.763817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.763697 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eae52ac-f711-47a1-b7c7-2c324a856890-configmap-metrics-client-ca\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:10:56.920999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920909 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" exitCode=0 Jul 07 17:10:56.920999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920940 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" exitCode=0 Jul 07 17:10:56.920999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920949 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" exitCode=0 Jul 07 17:10:56.920999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920957 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" exitCode=0 Jul 07 17:10:56.920999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920965 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" exitCode=0 Jul 07 17:10:56.920999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920973 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" exitCode=0 Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.920997 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921076 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921013 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921113 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921140 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921166 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921184 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921198 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} Jul 07 17:10:56.921280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.921223 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4eae52ac-f711-47a1-b7c7-2c324a856890","Type":"ContainerDied","Data":"d21e381be33a0bb3c972f8faf1cc98cf0fb4b448b7e64d59432e9907bd17a695"} Jul 07 17:10:56.929470 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.929419 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.938020 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.937999 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.944774 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.944756 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.952268 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.952244 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:10:56.952766 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.952746 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.959406 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.959389 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.964698 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.964674 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:10:56.971033 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.970988 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.978594 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.978550 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.978912 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.978881 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.978999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.978927 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} err="failed to get container status \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" Jul 07 17:10:56.979053 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979002 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.979355 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.979317 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.979400 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979375 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} err="failed to get container status \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" Jul 07 17:10:56.979440 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979402 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.979725 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.979703 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.979808 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979728 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} err="failed to get container status \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" Jul 07 17:10:56.979808 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979748 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.979976 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.979959 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.980028 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979979 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} err="failed to get container status \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" Jul 07 17:10:56.980028 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.979992 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.980239 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.980223 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.980287 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980243 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} err="failed to get container status \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" Jul 07 17:10:56.980287 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980256 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.980467 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.980452 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.980507 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980471 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} err="failed to get container status \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" Jul 07 17:10:56.980507 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980485 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.980707 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:10:56.980693 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.980772 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980709 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} err="failed to get container status \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" Jul 07 17:10:56.980772 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980720 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.980948 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980929 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} err="failed to get container status \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" Jul 07 17:10:56.980995 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.980949 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.981191 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981170 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} err="failed to get container status \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" Jul 07 17:10:56.981256 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981193 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.981394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981378 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} err="failed to get container status \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" Jul 07 17:10:56.981434 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981396 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.981628 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981607 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} err="failed to get container status \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" Jul 07 17:10:56.981628 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981626 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.981820 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981804 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} err="failed to get container status \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" Jul 07 17:10:56.981820 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.981819 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.982046 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982017 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} err="failed to get container status \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" Jul 07 17:10:56.982087 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982056 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.982270 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982252 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} err="failed to get container status \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" Jul 07 17:10:56.982313 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982270 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.982467 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982452 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} err="failed to get container status \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" Jul 07 17:10:56.982511 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982467 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.982696 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982681 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} err="failed to get container status \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" Jul 07 17:10:56.982696 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982695 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.982911 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982893 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} err="failed to get container status \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" Jul 07 17:10:56.982954 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.982912 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.983137 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983118 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} err="failed to get container status \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" Jul 07 17:10:56.983137 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983136 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.983333 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983317 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} err="failed to get container status \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" Jul 07 17:10:56.983333 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983332 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.983521 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983503 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} err="failed to get container status \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" Jul 07 17:10:56.983564 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983522 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.983725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983708 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} err="failed to get container status \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" Jul 07 17:10:56.983765 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983725 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.983924 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983908 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} err="failed to get container status \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" Jul 07 17:10:56.983924 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.983923 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.984127 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984113 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} err="failed to get container status \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" Jul 07 17:10:56.984160 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984127 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.984315 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984300 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} err="failed to get container status \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" Jul 07 17:10:56.984351 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984316 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.984505 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984488 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} err="failed to get container status \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" Jul 07 17:10:56.984538 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984506 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.984725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984706 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} err="failed to get container status \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" Jul 07 17:10:56.984782 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984726 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.984907 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984892 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} err="failed to get container status \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" Jul 07 17:10:56.984960 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.984907 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.985119 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985102 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} err="failed to get container status \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" Jul 07 17:10:56.985119 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985118 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.985294 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985279 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} err="failed to get container status \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" Jul 07 17:10:56.985336 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985294 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.985487 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985471 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} err="failed to get container status \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" Jul 07 17:10:56.985487 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985486 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.985670 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985651 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} err="failed to get container status \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" Jul 07 17:10:56.985716 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985672 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.985818 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985802 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} err="failed to get container status \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" Jul 07 17:10:56.985852 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.985820 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.986023 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986005 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} err="failed to get container status \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" Jul 07 17:10:56.986023 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986024 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.986230 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986213 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} err="failed to get container status \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" Jul 07 17:10:56.986284 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986230 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.986406 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986386 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} err="failed to get container status \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" Jul 07 17:10:56.986469 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986410 2582 scope.go:117] "RemoveContainer" containerID="983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4" Jul 07 17:10:56.986636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986617 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4"} err="failed to get container status \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": rpc error: code = NotFound desc = could not find container \"983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4\": container with ID starting with 983040c781a85a7e9f0ec1ab543670f81c1a220c87d7ca1ae4befccde29616a4 not found: ID does not exist" Jul 07 17:10:56.986694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986637 2582 scope.go:117] "RemoveContainer" containerID="7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e" Jul 07 17:10:56.986832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986816 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e"} err="failed to get container status \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": rpc error: code = NotFound desc = could not find container \"7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e\": container with ID starting with 7317bbf36430be32cfc5e5555730fab2aa061a03b3eafeddb4ab6cc97d558e5e not found: ID does not exist" Jul 07 17:10:56.986871 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.986832 2582 scope.go:117] "RemoveContainer" containerID="8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f" Jul 07 17:10:56.987033 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987017 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f"} err="failed to get container status \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": rpc error: code = NotFound desc = could not find container \"8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f\": container with ID starting with 8f8fb2ef6397e0285b1def55a7b9c5f8da9781124d73433cc3850b2e48b1cd3f not found: ID does not exist" Jul 07 17:10:56.987070 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987032 2582 scope.go:117] "RemoveContainer" containerID="df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5" Jul 07 17:10:56.987240 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987223 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5"} err="failed to get container status \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": rpc error: code = NotFound desc = could not find container \"df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5\": container with ID starting with df955197399c9f7248db6410a5992c49bcbdea025add211b55ba7e1ce2c45df5 not found: ID does not exist" Jul 07 17:10:56.987277 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987240 2582 scope.go:117] "RemoveContainer" containerID="29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4" Jul 07 17:10:56.987415 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987397 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4"} err="failed to get container status \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": rpc error: code = NotFound desc = could not find container \"29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4\": container with ID starting with 29c3882182f9ab1531fd5b96b1d7158e0c5e1b676e51c907df36050a3c7a59f4 not found: ID does not exist" Jul 07 17:10:56.987455 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987415 2582 scope.go:117] "RemoveContainer" containerID="a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe" Jul 07 17:10:56.987657 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987638 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe"} err="failed to get container status \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": rpc error: code = NotFound desc = could not find container \"a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe\": container with ID starting with a3774b90ee3f69ffffd4428bfce3622474362b3c54f8ba388c3deeca2cc29abe not found: ID does not exist" Jul 07 17:10:56.987710 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987658 2582 scope.go:117] "RemoveContainer" containerID="1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a" Jul 07 17:10:56.987899 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:56.987875 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a"} err="failed to get container status \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": rpc error: code = NotFound desc = could not find container \"1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a\": container with ID starting with 1c0832c6bd5aef0a20cb8e8a8e686a46127a531b31f6ac162ea5691426532f9a not found: ID does not exist" Jul 07 17:10:57.006786 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.006764 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:10:57.007103 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007091 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="prometheus" Jul 07 17:10:57.007145 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007105 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="prometheus" Jul 07 17:10:57.007145 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007115 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-web" Jul 07 17:10:57.007145 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007122 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-web" Jul 07 17:10:57.007145 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007130 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="config-reloader" Jul 07 17:10:57.007145 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007135 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="config-reloader" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007149 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-thanos" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007155 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-thanos" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007163 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" containerName="registry" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007169 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" containerName="registry" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007174 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007180 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007187 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="thanos-sidecar" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007193 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="thanos-sidecar" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007202 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="init-config-reloader" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007207 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="init-config-reloader" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007259 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="config-reloader" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007266 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e59b48ea-ba2f-4eb5-b74c-1f246b8465fb" containerName="registry" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007272 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-web" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007279 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="thanos-sidecar" Jul 07 17:10:57.007282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007285 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy-thanos" Jul 07 17:10:57.007723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007291 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="kube-rbac-proxy" Jul 07 17:10:57.007723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.007296 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" containerName="prometheus" Jul 07 17:10:57.012620 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.012604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.018623 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.018603 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 17:10:57.019237 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.019215 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 17:10:57.019622 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.019602 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 17:10:57.019724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.019619 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 17:10:57.019827 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.019716 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 17:10:57.019827 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.019799 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kx5xw\"" Jul 07 17:10:57.019980 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.019931 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 17:10:57.020192 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.020176 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 17:10:57.020252 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.020176 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 17:10:57.020549 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.020534 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 17:10:57.020647 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.020598 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 17:10:57.020647 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.020621 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 17:10:57.021091 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.021077 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-donmeildpvgpf\"" Jul 07 17:10:57.022892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.022877 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 17:10:57.024825 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.024807 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 17:10:57.025163 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.025147 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 17:10:57.030927 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.030906 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:10:57.068045 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbd28042-1af8-4973-aea3-d27764a33de4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068198 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068198 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068198 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068198 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068198 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068198 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068184 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068202 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068221 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068276 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068302 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068324 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068387 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-config\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbd28042-1af8-4973-aea3-d27764a33de4-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7tv5\" (UniqueName: \"kubernetes.io/projected/fbd28042-1af8-4973-aea3-d27764a33de4-kube-api-access-t7tv5\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068486 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.068637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.068523 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169248 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169248 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-config\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169480 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbd28042-1af8-4973-aea3-d27764a33de4-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169480 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169480 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169433 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7tv5\" (UniqueName: \"kubernetes.io/projected/fbd28042-1af8-4973-aea3-d27764a33de4-kube-api-access-t7tv5\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169480 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169454 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169480 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbd28042-1af8-4973-aea3-d27764a33de4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169658 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169685 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169738 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.169832 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.170282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.170282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169868 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.170282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.170282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169955 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.170282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.169989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.170282 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.170199 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.171351 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.170982 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.171351 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.171259 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.172082 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.172311 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.172524 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-config\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.172727 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.172909 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbd28042-1af8-4973-aea3-d27764a33de4-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.173011 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.173366 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.173541 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.173947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.173666 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.174593 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.174557 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbd28042-1af8-4973-aea3-d27764a33de4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.174663 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.174555 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.174725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.174681 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.174811 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.174794 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.175384 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.175368 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbd28042-1af8-4973-aea3-d27764a33de4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.175755 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.175741 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbd28042-1af8-4973-aea3-d27764a33de4-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.181817 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.181801 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7tv5\" (UniqueName: \"kubernetes.io/projected/fbd28042-1af8-4973-aea3-d27764a33de4-kube-api-access-t7tv5\") pod \"prometheus-k8s-0\" (UID: \"fbd28042-1af8-4973-aea3-d27764a33de4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.322299 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.322265 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:10:57.467771 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.467691 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 17:10:57.471306 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:10:57.471281 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd28042_1af8_4973_aea3_d27764a33de4.slice/crio-b61e0bea4fa1dc2cfe914290d225f4f395d22628acb33dc9249f45f6c4322b44 WatchSource:0}: Error finding container b61e0bea4fa1dc2cfe914290d225f4f395d22628acb33dc9249f45f6c4322b44: Status 404 returned error can't find the container with id b61e0bea4fa1dc2cfe914290d225f4f395d22628acb33dc9249f45f6c4322b44 Jul 07 17:10:57.927589 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.927539 2582 generic.go:356] "Generic (PLEG): container finished" podID="fbd28042-1af8-4973-aea3-d27764a33de4" containerID="7c43c578127ce4c1324913866d206862480e36f2da35045fadf2d41e9152a8b0" exitCode=0 Jul 07 17:10:57.927750 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.927622 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerDied","Data":"7c43c578127ce4c1324913866d206862480e36f2da35045fadf2d41e9152a8b0"} Jul 07 17:10:57.927750 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:57.927687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"b61e0bea4fa1dc2cfe914290d225f4f395d22628acb33dc9249f45f6c4322b44"} Jul 07 17:10:58.934876 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.934837 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eae52ac-f711-47a1-b7c7-2c324a856890" path="/var/lib/kubelet/pods/4eae52ac-f711-47a1-b7c7-2c324a856890/volumes" Jul 07 17:10:58.936357 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.936331 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"07ebd4c809a1f5546f3de7fcb16ed6147d87e933d0674faeda34016a6ae3309b"} Jul 07 17:10:58.936442 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.936392 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"70284242add6be35aa07f15b115bc9198865069926516684a8fd7802ba5fefea"} Jul 07 17:10:58.936442 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.936422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"3bf41e1e9ac6b3bc44cc68785ec2e1aa79e39cc06d9639b54bca6572c20a718b"} Jul 07 17:10:58.936503 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.936448 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"c1dd84711f897d2eb110b53066d736e581275658d0f5b12749c6dbfa66362705"} Jul 07 17:10:58.936503 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.936471 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"f8141eaf7ed67532886f9ed7f8c8b307ee050e118de59ffab3f927e250f1d673"} Jul 07 17:10:58.936503 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.936495 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd28042-1af8-4973-aea3-d27764a33de4","Type":"ContainerStarted","Data":"e6f793c05f6ee637eab925008f052dad11c026287314f78173e356e3d9e10a30"} Jul 07 17:10:58.971131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:10:58.971069 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.951130916 podStartE2EDuration="2.971049677s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.918761ms" isStatefulPod=false podCreationTimestamp="2026-07-07 17:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:10:58.967613558 +0000 UTC m=+210.827693632" watchObservedRunningTime="2026-07-07 17:10:58.971049677 +0000 UTC m=+210.831129750" Jul 07 17:11:02.322715 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:11:02.322667 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:11:57.323327 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:11:57.323226 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:11:57.339226 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:11:57.339183 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:11:58.190690 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:11:58.190662 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 17:12:28.611490 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:12:28.611458 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:12:28.627479 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:12:28.627450 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:12:28.628821 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:12:28.628789 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 17:13:10.583393 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.583358 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7"] Jul 07 17:13:10.586841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.586824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.591460 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.591435 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 17:13:10.591754 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.591738 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 17:13:10.591965 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.591949 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kchrk\"" Jul 07 17:13:10.596066 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.596045 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 17:13:10.596440 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.596428 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 17:13:10.606047 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.606025 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7"] Jul 07 17:13:10.694744 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.694698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8119380d-de5a-46dd-a204-ca61cc80e57b-webhook-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.694937 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.694787 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8119380d-de5a-46dd-a204-ca61cc80e57b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.694937 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.694834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj68c\" (UniqueName: \"kubernetes.io/projected/8119380d-de5a-46dd-a204-ca61cc80e57b-kube-api-access-lj68c\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.694937 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.694875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8119380d-de5a-46dd-a204-ca61cc80e57b-tmp\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.694937 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.694914 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8119380d-de5a-46dd-a204-ca61cc80e57b-manifests\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.795734 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.795672 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8119380d-de5a-46dd-a204-ca61cc80e57b-tmp\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.795734 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.795744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8119380d-de5a-46dd-a204-ca61cc80e57b-manifests\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.796007 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.795828 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8119380d-de5a-46dd-a204-ca61cc80e57b-webhook-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.796007 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.795876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8119380d-de5a-46dd-a204-ca61cc80e57b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.796007 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.795918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj68c\" (UniqueName: \"kubernetes.io/projected/8119380d-de5a-46dd-a204-ca61cc80e57b-kube-api-access-lj68c\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.796239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.796210 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8119380d-de5a-46dd-a204-ca61cc80e57b-manifests\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.798012 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.797983 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8119380d-de5a-46dd-a204-ca61cc80e57b-tmp\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.798257 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.798236 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8119380d-de5a-46dd-a204-ca61cc80e57b-webhook-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.798319 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.798282 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8119380d-de5a-46dd-a204-ca61cc80e57b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.817092 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.817055 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj68c\" (UniqueName: \"kubernetes.io/projected/8119380d-de5a-46dd-a204-ca61cc80e57b-kube-api-access-lj68c\") pod \"opendatahub-operator-controller-manager-547959ffc4-ktth7\" (UID: \"8119380d-de5a-46dd-a204-ca61cc80e57b\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:10.896913 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:10.896877 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:11.037238 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:11.037199 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7"] Jul 07 17:13:11.040376 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:13:11.040351 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8119380d_de5a_46dd_a204_ca61cc80e57b.slice/crio-6287e9bc847255cbf9f9d3ef6057b2de55e148bdecefd849c20f7d69aa5b3e5d WatchSource:0}: Error finding container 6287e9bc847255cbf9f9d3ef6057b2de55e148bdecefd849c20f7d69aa5b3e5d: Status 404 returned error can't find the container with id 6287e9bc847255cbf9f9d3ef6057b2de55e148bdecefd849c20f7d69aa5b3e5d Jul 07 17:13:11.042118 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:11.042102 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:13:11.472121 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:11.471105 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" event={"ID":"8119380d-de5a-46dd-a204-ca61cc80e57b","Type":"ContainerStarted","Data":"6287e9bc847255cbf9f9d3ef6057b2de55e148bdecefd849c20f7d69aa5b3e5d"} Jul 07 17:13:15.493105 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:15.493057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" event={"ID":"8119380d-de5a-46dd-a204-ca61cc80e57b","Type":"ContainerStarted","Data":"6bf423dab08d00821948bd00db0b66cdc72a99abf1ddf3baa0dc64eb3d075f03"} Jul 07 17:13:16.498595 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:16.498543 2582 generic.go:356] "Generic (PLEG): container finished" podID="8119380d-de5a-46dd-a204-ca61cc80e57b" containerID="6bf423dab08d00821948bd00db0b66cdc72a99abf1ddf3baa0dc64eb3d075f03" exitCode=0 Jul 07 17:13:16.498986 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:16.498632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" event={"ID":"8119380d-de5a-46dd-a204-ca61cc80e57b","Type":"ContainerDied","Data":"6bf423dab08d00821948bd00db0b66cdc72a99abf1ddf3baa0dc64eb3d075f03"} Jul 07 17:13:17.506232 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:17.506196 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" event={"ID":"8119380d-de5a-46dd-a204-ca61cc80e57b","Type":"ContainerStarted","Data":"0ad4e0be73d90df89ab0cacf505ff6ff8ac43fc2ce29770ef25e38f12adc338e"} Jul 07 17:13:17.506715 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:17.506278 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:17.537430 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:17.537380 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" podStartSLOduration=2.506738001 podStartE2EDuration="7.537363909s" totalImagesPullingTime="4.006387694s" totalInitContainerRuntime="1.024238214s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:17.534859927 +0000 UTC m=+349.394939999" watchObservedRunningTime="2026-07-07 17:13:17.537363909 +0000 UTC m=+349.397444069" Jul 07 17:13:28.512168 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:28.512085 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-ktth7" Jul 07 17:13:31.075647 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.075606 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v"] Jul 07 17:13:31.083903 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.083884 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.087403 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.087376 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 17:13:31.087537 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.087469 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sh26c\"" Jul 07 17:13:31.087633 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.087589 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 17:13:31.087738 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.087723 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 17:13:31.088518 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.088499 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 17:13:31.110995 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.110970 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v"] Jul 07 17:13:31.189446 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.189416 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbwh\" (UniqueName: \"kubernetes.io/projected/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-kube-api-access-swbwh\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.189633 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.189457 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-tls-certs\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.189633 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.189509 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-tmp\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.290280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.290245 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbwh\" (UniqueName: \"kubernetes.io/projected/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-kube-api-access-swbwh\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.290280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.290281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-tls-certs\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.290526 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.290305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-tmp\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.292614 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.292586 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-tmp\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.292818 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.292797 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-tls-certs\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.313719 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.313695 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbwh\" (UniqueName: \"kubernetes.io/projected/3cb9abe8-8b1e-4b48-93ae-b043104a9c43-kube-api-access-swbwh\") pod \"kube-auth-proxy-76676f7fb7-4wv8v\" (UID: \"3cb9abe8-8b1e-4b48-93ae-b043104a9c43\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.393153 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.393129 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" Jul 07 17:13:31.559520 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.559481 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v"] Jul 07 17:13:31.562705 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:13:31.562671 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb9abe8_8b1e_4b48_93ae_b043104a9c43.slice/crio-75e0ba28d277d4ce8d2a062e0e63c22f6f7f4383abf146540b17467e2b016b2a WatchSource:0}: Error finding container 75e0ba28d277d4ce8d2a062e0e63c22f6f7f4383abf146540b17467e2b016b2a: Status 404 returned error can't find the container with id 75e0ba28d277d4ce8d2a062e0e63c22f6f7f4383abf146540b17467e2b016b2a Jul 07 17:13:31.569893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:31.569856 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" event={"ID":"3cb9abe8-8b1e-4b48-93ae-b043104a9c43","Type":"ContainerStarted","Data":"75e0ba28d277d4ce8d2a062e0e63c22f6f7f4383abf146540b17467e2b016b2a"} Jul 07 17:13:35.591120 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:35.591084 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" event={"ID":"3cb9abe8-8b1e-4b48-93ae-b043104a9c43","Type":"ContainerStarted","Data":"a65be5597a699d41ad592b362932fe4979f86e993e997e92d40a19dffa4975f5"} Jul 07 17:13:35.613081 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:35.613022 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-4wv8v" podStartSLOduration=1.16869287 podStartE2EDuration="4.613007167s" totalImagesPullingTime="3.444314297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:35.611678413 +0000 UTC m=+367.471758486" watchObservedRunningTime="2026-07-07 17:13:35.613007167 +0000 UTC m=+367.473087278" Jul 07 17:13:38.859501 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.859469 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln"] Jul 07 17:13:38.863089 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.863048 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.867157 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.867136 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 17:13:38.867291 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.867250 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-t9hjr\"" Jul 07 17:13:38.886824 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.886791 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln"] Jul 07 17:13:38.953558 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953518 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953753 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953565 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953753 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953753 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953702 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953753 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953736 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953923 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953762 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953923 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953848 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmkf2\" (UniqueName: \"kubernetes.io/projected/28d6ce2b-c497-4b33-b5a2-2447b66361f5-kube-api-access-hmkf2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.953923 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:38.954052 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:38.953974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055343 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055527 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055527 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055393 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055527 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055527 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055780 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055540 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055780 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055780 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmkf2\" (UniqueName: \"kubernetes.io/projected/28d6ce2b-c497-4b33-b5a2-2447b66361f5-kube-api-access-hmkf2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055780 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055775 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055979 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055907 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.055979 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.055971 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.056079 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.056022 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.056079 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.056030 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.056269 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.056248 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.057903 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.057872 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.058176 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.058156 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.087392 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.087359 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d6ce2b-c497-4b33-b5a2-2447b66361f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.087756 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.087737 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmkf2\" (UniqueName: \"kubernetes.io/projected/28d6ce2b-c497-4b33-b5a2-2447b66361f5-kube-api-access-hmkf2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45rdln\" (UID: \"28d6ce2b-c497-4b33-b5a2-2447b66361f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.173300 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.173260 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:39.348717 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.348686 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln"] Jul 07 17:13:39.349606 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:13:39.349568 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d6ce2b_c497_4b33_b5a2_2447b66361f5.slice/crio-f4deca8276cd2cce98081ce1c8d639cad05a2c8d6d82bba328865e88e248516e WatchSource:0}: Error finding container f4deca8276cd2cce98081ce1c8d639cad05a2c8d6d82bba328865e88e248516e: Status 404 returned error can't find the container with id f4deca8276cd2cce98081ce1c8d639cad05a2c8d6d82bba328865e88e248516e Jul 07 17:13:39.610091 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.610060 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" event={"ID":"28d6ce2b-c497-4b33-b5a2-2447b66361f5","Type":"ContainerStarted","Data":"f4deca8276cd2cce98081ce1c8d639cad05a2c8d6d82bba328865e88e248516e"} Jul 07 17:13:39.736471 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.736433 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v"] Jul 07 17:13:39.741169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.741146 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.744254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.744231 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 17:13:39.745654 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.745634 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7gz65\"" Jul 07 17:13:39.745763 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.745653 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:13:39.745831 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.745815 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 17:13:39.746096 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.746081 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 17:13:39.747171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.747133 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 17:13:39.755280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.755256 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v"] Jul 07 17:13:39.762956 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.762933 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/139ed07a-0356-426c-bdb2-0e4c166a916c-manager-config\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.762956 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.762959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/139ed07a-0356-426c-bdb2-0e4c166a916c-metrics-cert\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.763130 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.762992 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5z82\" (UniqueName: \"kubernetes.io/projected/139ed07a-0356-426c-bdb2-0e4c166a916c-kube-api-access-g5z82\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.763130 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.763106 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/139ed07a-0356-426c-bdb2-0e4c166a916c-cert\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.863604 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.863508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/139ed07a-0356-426c-bdb2-0e4c166a916c-cert\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.863604 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.863563 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/139ed07a-0356-426c-bdb2-0e4c166a916c-manager-config\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.864111 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.863606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/139ed07a-0356-426c-bdb2-0e4c166a916c-metrics-cert\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.864111 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.863659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5z82\" (UniqueName: \"kubernetes.io/projected/139ed07a-0356-426c-bdb2-0e4c166a916c-kube-api-access-g5z82\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.864308 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.864282 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/139ed07a-0356-426c-bdb2-0e4c166a916c-manager-config\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.866059 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.866029 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/139ed07a-0356-426c-bdb2-0e4c166a916c-cert\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.866163 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.866128 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/139ed07a-0356-426c-bdb2-0e4c166a916c-metrics-cert\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:39.878224 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:39.878204 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5z82\" (UniqueName: \"kubernetes.io/projected/139ed07a-0356-426c-bdb2-0e4c166a916c-kube-api-access-g5z82\") pod \"lws-controller-manager-5bbf97565b-2hm9v\" (UID: \"139ed07a-0356-426c-bdb2-0e4c166a916c\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:40.051317 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:40.051287 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:40.345055 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:40.344778 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v"] Jul 07 17:13:40.347728 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:13:40.347694 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139ed07a_0356_426c_bdb2_0e4c166a916c.slice/crio-c02253899bb80c625f142a27427864c4748593c2488006633b2050c3a0105fbb WatchSource:0}: Error finding container c02253899bb80c625f142a27427864c4748593c2488006633b2050c3a0105fbb: Status 404 returned error can't find the container with id c02253899bb80c625f142a27427864c4748593c2488006633b2050c3a0105fbb Jul 07 17:13:40.617831 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:40.617789 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" event={"ID":"139ed07a-0356-426c-bdb2-0e4c166a916c","Type":"ContainerStarted","Data":"c02253899bb80c625f142a27427864c4748593c2488006633b2050c3a0105fbb"} Jul 07 17:13:43.512542 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:43.512501 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:13:43.512865 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:43.512591 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:13:43.512865 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:43.512623 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:13:43.636448 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:43.636418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" event={"ID":"28d6ce2b-c497-4b33-b5a2-2447b66361f5","Type":"ContainerStarted","Data":"f913ed0eeb66f5bb59cfec20b41733db68e0c341fcf6578b66cec36f80c23b38"} Jul 07 17:13:43.717002 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:43.716954 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" podStartSLOduration=1.556303601 podStartE2EDuration="5.716938067s" totalImagesPullingTime="4.160634466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:43.708305778 +0000 UTC m=+375.568385851" watchObservedRunningTime="2026-07-07 17:13:43.716938067 +0000 UTC m=+375.577018138" Jul 07 17:13:44.174289 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.174258 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:44.178881 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.178859 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:44.643761 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.643727 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" event={"ID":"139ed07a-0356-426c-bdb2-0e4c166a916c","Type":"ContainerStarted","Data":"78ab939ece380c654daed24e6a50a51a102f7c7cdb326222b90390d7a66f4c02"} Jul 07 17:13:44.644205 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.643804 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:44.644205 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.643916 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:13:44.644874 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.644857 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45rdln" Jul 07 17:13:44.702664 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:44.702600 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" podStartSLOduration=2.465401248 podStartE2EDuration="5.702565308s" totalImagesPullingTime="3.23716406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:44.700443656 +0000 UTC m=+376.560523729" watchObservedRunningTime="2026-07-07 17:13:44.702565308 +0000 UTC m=+376.562645383" Jul 07 17:13:46.102859 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.102824 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z5btm"] Jul 07 17:13:46.106984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.106959 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.112671 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.112645 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z5btm"] Jul 07 17:13:46.115116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.115092 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 17:13:46.115901 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.115884 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4jvhh\"" Jul 07 17:13:46.140945 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.140912 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd"] Jul 07 17:13:46.144299 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.144273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.161155 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.161126 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cvsn6\"" Jul 07 17:13:46.162530 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.162514 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 17:13:46.197620 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.197585 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd"] Jul 07 17:13:46.238174 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.238144 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8sq\" (UniqueName: \"kubernetes.io/projected/0b9b46ee-c52a-4b30-a95b-96c85e84c51b-kube-api-access-js8sq\") pod \"kserve-controller-manager-d6f4b65b8-z5btm\" (UID: \"0b9b46ee-c52a-4b30-a95b-96c85e84c51b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.238329 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.238196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khkzw\" (UniqueName: \"kubernetes.io/projected/61271ea3-07a6-4510-bca4-5bfdf8616073-kube-api-access-khkzw\") pod \"llmisvc-controller-manager-777c6bd7d5-zcmnd\" (UID: \"61271ea3-07a6-4510-bca4-5bfdf8616073\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.238329 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.238274 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b9b46ee-c52a-4b30-a95b-96c85e84c51b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z5btm\" (UID: \"0b9b46ee-c52a-4b30-a95b-96c85e84c51b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.238329 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.238306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61271ea3-07a6-4510-bca4-5bfdf8616073-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zcmnd\" (UID: \"61271ea3-07a6-4510-bca4-5bfdf8616073\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.339492 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.339457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8sq\" (UniqueName: \"kubernetes.io/projected/0b9b46ee-c52a-4b30-a95b-96c85e84c51b-kube-api-access-js8sq\") pod \"kserve-controller-manager-d6f4b65b8-z5btm\" (UID: \"0b9b46ee-c52a-4b30-a95b-96c85e84c51b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.339492 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.339501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkzw\" (UniqueName: \"kubernetes.io/projected/61271ea3-07a6-4510-bca4-5bfdf8616073-kube-api-access-khkzw\") pod \"llmisvc-controller-manager-777c6bd7d5-zcmnd\" (UID: \"61271ea3-07a6-4510-bca4-5bfdf8616073\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.339799 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.339537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b9b46ee-c52a-4b30-a95b-96c85e84c51b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z5btm\" (UID: \"0b9b46ee-c52a-4b30-a95b-96c85e84c51b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.339799 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.339564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61271ea3-07a6-4510-bca4-5bfdf8616073-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zcmnd\" (UID: \"61271ea3-07a6-4510-bca4-5bfdf8616073\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.342092 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.342058 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b9b46ee-c52a-4b30-a95b-96c85e84c51b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z5btm\" (UID: \"0b9b46ee-c52a-4b30-a95b-96c85e84c51b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.342092 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.342070 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61271ea3-07a6-4510-bca4-5bfdf8616073-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zcmnd\" (UID: \"61271ea3-07a6-4510-bca4-5bfdf8616073\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.356097 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.356042 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8sq\" (UniqueName: \"kubernetes.io/projected/0b9b46ee-c52a-4b30-a95b-96c85e84c51b-kube-api-access-js8sq\") pod \"kserve-controller-manager-d6f4b65b8-z5btm\" (UID: \"0b9b46ee-c52a-4b30-a95b-96c85e84c51b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.356209 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.356185 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khkzw\" (UniqueName: \"kubernetes.io/projected/61271ea3-07a6-4510-bca4-5bfdf8616073-kube-api-access-khkzw\") pod \"llmisvc-controller-manager-777c6bd7d5-zcmnd\" (UID: \"61271ea3-07a6-4510-bca4-5bfdf8616073\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.418659 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.418631 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:46.453980 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.453934 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:46.596283 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.596232 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z5btm"] Jul 07 17:13:46.600374 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:13:46.600318 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9b46ee_c52a_4b30_a95b_96c85e84c51b.slice/crio-d57f15f2f55802b74b10648f8699f06840b7e70b16ea5a801f15b0d3c3b0706c WatchSource:0}: Error finding container d57f15f2f55802b74b10648f8699f06840b7e70b16ea5a801f15b0d3c3b0706c: Status 404 returned error can't find the container with id d57f15f2f55802b74b10648f8699f06840b7e70b16ea5a801f15b0d3c3b0706c Jul 07 17:13:46.655057 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.655017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" event={"ID":"0b9b46ee-c52a-4b30-a95b-96c85e84c51b","Type":"ContainerStarted","Data":"d57f15f2f55802b74b10648f8699f06840b7e70b16ea5a801f15b0d3c3b0706c"} Jul 07 17:13:46.660517 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:46.660492 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd"] Jul 07 17:13:46.662646 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:13:46.662621 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61271ea3_07a6_4510_bca4_5bfdf8616073.slice/crio-663b8d04f581230486c581f0344d682f705ba16fee007bc88f60209efd2f7eb8 WatchSource:0}: Error finding container 663b8d04f581230486c581f0344d682f705ba16fee007bc88f60209efd2f7eb8: Status 404 returned error can't find the container with id 663b8d04f581230486c581f0344d682f705ba16fee007bc88f60209efd2f7eb8 Jul 07 17:13:47.660789 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:47.660756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" event={"ID":"61271ea3-07a6-4510-bca4-5bfdf8616073","Type":"ContainerStarted","Data":"663b8d04f581230486c581f0344d682f705ba16fee007bc88f60209efd2f7eb8"} Jul 07 17:13:49.672499 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:49.672449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" event={"ID":"0b9b46ee-c52a-4b30-a95b-96c85e84c51b","Type":"ContainerStarted","Data":"21172d28c6df3eaf99a77082786c2552b0dd25c3dac1f7809a9c5aca93cacc9d"} Jul 07 17:13:49.672916 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:49.672601 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:13:49.710139 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:49.710081 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" podStartSLOduration=0.803020818 podStartE2EDuration="3.71006626s" totalImagesPullingTime="2.907045442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:49.709253283 +0000 UTC m=+381.569333366" watchObservedRunningTime="2026-07-07 17:13:49.71006626 +0000 UTC m=+381.570146331" Jul 07 17:13:51.683086 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:51.683045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" event={"ID":"61271ea3-07a6-4510-bca4-5bfdf8616073","Type":"ContainerStarted","Data":"54616a70ae6cceca964939b0d2095d1bc47720b50e79f73709d54b6eaa6405f7"} Jul 07 17:13:51.683502 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:51.683115 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:13:51.704497 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:51.704452 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" podStartSLOduration=0.883193345 podStartE2EDuration="5.704436529s" totalImagesPullingTime="4.821243184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:51.703882266 +0000 UTC m=+383.563962338" watchObservedRunningTime="2026-07-07 17:13:51.704436529 +0000 UTC m=+383.564516638" Jul 07 17:13:55.650735 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:13:55.650699 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-2hm9v" Jul 07 17:14:20.683005 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:14:20.682977 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z5btm" Jul 07 17:14:22.689758 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:14:22.689725 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zcmnd" Jul 07 17:16:12.682384 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.682353 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:16:12.686012 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.685994 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.692371 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.692343 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vv4vg\"" Jul 07 17:16:12.692529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.692348 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 17:16:12.693189 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.693169 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 17:16:12.693267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.693173 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 17:16:12.698740 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.698721 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:16:12.751357 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.751318 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:16:12.774204 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.774169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6460b40e-7a6f-4e03-952b-d81aaabcacb0-config-file\") pod \"limitador-limitador-54d6d988d5-xpt95\" (UID: \"6460b40e-7a6f-4e03-952b-d81aaabcacb0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.774366 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.774226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpts7\" (UniqueName: \"kubernetes.io/projected/6460b40e-7a6f-4e03-952b-d81aaabcacb0-kube-api-access-gpts7\") pod \"limitador-limitador-54d6d988d5-xpt95\" (UID: \"6460b40e-7a6f-4e03-952b-d81aaabcacb0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.875066 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.875026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpts7\" (UniqueName: \"kubernetes.io/projected/6460b40e-7a6f-4e03-952b-d81aaabcacb0-kube-api-access-gpts7\") pod \"limitador-limitador-54d6d988d5-xpt95\" (UID: \"6460b40e-7a6f-4e03-952b-d81aaabcacb0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.875241 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.875151 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6460b40e-7a6f-4e03-952b-d81aaabcacb0-config-file\") pod \"limitador-limitador-54d6d988d5-xpt95\" (UID: \"6460b40e-7a6f-4e03-952b-d81aaabcacb0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.875825 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.875805 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6460b40e-7a6f-4e03-952b-d81aaabcacb0-config-file\") pod \"limitador-limitador-54d6d988d5-xpt95\" (UID: \"6460b40e-7a6f-4e03-952b-d81aaabcacb0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.884963 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.884939 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpts7\" (UniqueName: \"kubernetes.io/projected/6460b40e-7a6f-4e03-952b-d81aaabcacb0-kube-api-access-gpts7\") pod \"limitador-limitador-54d6d988d5-xpt95\" (UID: \"6460b40e-7a6f-4e03-952b-d81aaabcacb0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:12.996294 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:12.996207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:13.135542 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:13.135515 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:16:13.138148 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:16:13.138122 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6460b40e_7a6f_4e03_952b_d81aaabcacb0.slice/crio-348b4a72c70b9a4e674839b3a7371283e6cc6b12cadf81a1345b7fb08fcd5254 WatchSource:0}: Error finding container 348b4a72c70b9a4e674839b3a7371283e6cc6b12cadf81a1345b7fb08fcd5254: Status 404 returned error can't find the container with id 348b4a72c70b9a4e674839b3a7371283e6cc6b12cadf81a1345b7fb08fcd5254 Jul 07 17:16:13.327956 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:13.327861 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" event={"ID":"6460b40e-7a6f-4e03-952b-d81aaabcacb0","Type":"ContainerStarted","Data":"348b4a72c70b9a4e674839b3a7371283e6cc6b12cadf81a1345b7fb08fcd5254"} Jul 07 17:16:16.343484 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:16.343443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" event={"ID":"6460b40e-7a6f-4e03-952b-d81aaabcacb0","Type":"ContainerStarted","Data":"60a8acb06870b5342c2e11032a44e456934cf31b01f788afba449b6bea4f7120"} Jul 07 17:16:16.343906 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:16.343513 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:16:16.365059 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:16.364940 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" podStartSLOduration=1.9539286850000002 podStartE2EDuration="4.364919308s" totalImagesPullingTime="2.410990623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:16:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:16:16.363162906 +0000 UTC m=+528.223242979" watchObservedRunningTime="2026-07-07 17:16:16.364919308 +0000 UTC m=+528.224999382" Jul 07 17:16:27.349146 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:16:27.349063 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xpt95" Jul 07 17:17:27.674953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:17:27.674924 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:17:28.649943 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:17:28.649911 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:17:28.666441 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:17:28.666416 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:17:52.183834 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:17:52.183752 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:18:21.371307 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:18:21.371267 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:18:31.098630 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:18:31.098584 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:18:40.269697 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:18:40.269655 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:18:52.772243 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:18:52.772208 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:19:02.076283 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:19:02.076246 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:20:33.409233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:20:33.409192 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:20:42.871013 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:20:42.870975 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:20:53.392160 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:20:53.392061 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:21:02.074266 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:21:02.074231 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:21:13.475531 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:21:13.475494 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:22:20.479004 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:20.478968 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:22:28.689858 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:28.689823 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:22:28.701242 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:28.701215 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:22:31.380539 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:31.380499 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:22:40.684041 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:40.684009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:22:47.093691 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:47.093659 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:22:57.975226 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:22:57.975190 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:23:08.203041 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:23:08.203011 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:23:18.791069 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:23:18.791021 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:23:29.305032 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:23:29.304998 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:23:50.890621 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:23:50.890521 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:24:03.673291 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:24:03.673257 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:24:26.300624 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:24:26.300594 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:24:37.072349 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:24:37.072310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:24:52.280018 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:24:52.279983 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:25:01.668892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:25:01.668860 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:25:11.776543 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:25:11.776509 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:25:21.178820 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:25:21.178737 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:25:30.575106 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:25:30.575073 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:27:17.975267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:27:17.975234 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:27:27.570251 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:27:27.570212 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:27:28.733662 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:27:28.733631 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:27:28.737401 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:27:28.737379 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:27:41.077186 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:27:41.077144 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:27:51.979409 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:27:51.979363 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:28:03.474793 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:28:03.474756 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:28:13.479668 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:28:13.479632 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:28:23.672663 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:28:23.672563 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:28:33.973020 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:28:33.972988 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:28:45.583586 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:28:45.583533 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:28:53.582862 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:28:53.582821 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:29:04.896054 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:29:04.896017 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:29:28.086712 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:29:28.086677 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:29:38.180356 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:29:38.180325 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:29:48.383651 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:29:48.383616 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:29:58.607485 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:29:58.607405 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:30:08.379644 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:30:08.379604 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:30:21.286087 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:30:21.286051 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:30:35.181359 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:30:35.181324 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:30:44.688228 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:30:44.688194 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:30:52.484696 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:30:52.484665 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:31:03.732723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:31:03.732688 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:31:13.276651 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:31:13.276611 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:31:27.176038 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:31:27.175961 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:31:38.394547 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:31:38.394516 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:31:51.497035 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:31:51.497000 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:32:00.710548 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:00.710515 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:32:13.090104 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:13.090069 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:32:28.766475 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:28.766444 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:32:28.768940 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:28.768918 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:32:30.680481 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:30.680446 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:32:42.287058 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:42.287024 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:32:54.959731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:32:54.959648 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:33:08.389857 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:08.389819 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:33:17.236421 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:17.236383 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:33:29.139013 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:29.138978 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:33:35.682617 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.682554 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx"] Jul 07 17:33:35.686206 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.686185 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.689388 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.689362 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9d5662de-openshift-default-dockercfg-9prx7\"" Jul 07 17:33:35.725256 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.725219 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx"] Jul 07 17:33:35.796454 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-data\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796630 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796457 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-podinfo\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796630 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796503 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-credential-socket\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796630 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istiod-ca-cert\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796630 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796567 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-certs\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796773 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796635 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-token\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796773 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-socket\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796773 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796687 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxdmg\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-kube-api-access-qxdmg\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.796773 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.796708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-envoy\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897319 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897284 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-socket\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897319 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxdmg\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-kube-api-access-qxdmg\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897343 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-envoy\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-data\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-podinfo\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-credential-socket\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897472 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istiod-ca-cert\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-certs\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897875 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897538 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-token\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897875 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897762 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-socket\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897915 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-certs\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.897975 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.897906 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-credential-socket\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.898075 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.898004 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-data\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.898192 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.898168 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istiod-ca-cert\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.899703 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.899682 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-envoy\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.899941 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.899923 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-podinfo\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.912984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.912953 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-token\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.913703 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.913683 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxdmg\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-kube-api-access-qxdmg\") pod \"e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:35.997450 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:35.997387 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:36.150775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.150751 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx"] Jul 07 17:33:36.152879 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:33:36.152852 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8001123a_4ae2_4dc0_93d1_00d7ca27040e.slice/crio-dad83743448936a541d58db3e9aca08162f7cdd4562389e05ff038430f7659f8 WatchSource:0}: Error finding container dad83743448936a541d58db3e9aca08162f7cdd4562389e05ff038430f7659f8: Status 404 returned error can't find the container with id dad83743448936a541d58db3e9aca08162f7cdd4562389e05ff038430f7659f8 Jul 07 17:33:36.154961 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.154926 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:33:36.155044 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.154988 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:33:36.155044 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.155033 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:33:36.253807 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.253737 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" event={"ID":"8001123a-4ae2-4dc0-93d1-00d7ca27040e","Type":"ContainerStarted","Data":"d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38"} Jul 07 17:33:36.253807 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.253797 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" event={"ID":"8001123a-4ae2-4dc0-93d1-00d7ca27040e","Type":"ContainerStarted","Data":"dad83743448936a541d58db3e9aca08162f7cdd4562389e05ff038430f7659f8"} Jul 07 17:33:36.297816 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.297762 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" podStartSLOduration=1.297741688 podStartE2EDuration="1.297741688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:36.284886027 +0000 UTC m=+1568.144966103" watchObservedRunningTime="2026-07-07 17:33:36.297741688 +0000 UTC m=+1568.157821761" Jul 07 17:33:36.997780 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:36.997750 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:38.003369 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:38.003342 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:38.264444 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:38.264357 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:38.265244 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:38.265224 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:40.881795 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:40.881759 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:33:42.218017 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:42.217972 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx"] Jul 07 17:33:42.218499 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:42.218281 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" podUID="8001123a-4ae2-4dc0-93d1-00d7ca27040e" containerName="istio-proxy" containerID="cri-o://d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38" gracePeriod=30 Jul 07 17:33:48.867429 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.867401 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:48.918831 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.918729 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-podinfo\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.918831 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.918765 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istiod-ca-cert\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.918831 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.918798 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-data\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.918845 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxdmg\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-kube-api-access-qxdmg\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.918881 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-envoy\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919056 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-credential-socket\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919088 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-certs\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919125 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-socket\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919152 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-token\") pod \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\" (UID: \"8001123a-4ae2-4dc0-93d1-00d7ca27040e\") " Jul 07 17:33:48.919390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919156 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:33:48.919390 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919310 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:33:48.919553 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919386 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:33:48.919553 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919435 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:33:48.919694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919559 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-credential-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:48.919694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919595 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:48.919694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919610 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-workload-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:48.919694 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919623 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istiod-ca-cert\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:48.919991 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.919964 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-data" (OuterVolumeSpecName: "istio-data") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:33:48.921551 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.921524 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-token" (OuterVolumeSpecName: "istio-token") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:33:48.921736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.921637 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 17:33:48.921736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.921668 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-kube-api-access-qxdmg" (OuterVolumeSpecName: "kube-api-access-qxdmg") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "kube-api-access-qxdmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:33:48.921850 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:48.921789 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8001123a-4ae2-4dc0-93d1-00d7ca27040e" (UID: "8001123a-4ae2-4dc0-93d1-00d7ca27040e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:33:49.020068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.020032 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-podinfo\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:49.020068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.020060 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-data\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:49.020068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.020069 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxdmg\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-kube-api-access-qxdmg\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:49.020068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.020079 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-envoy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:49.020330 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.020088 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8001123a-4ae2-4dc0-93d1-00d7ca27040e-istio-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:33:49.318977 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.318893 2582 generic.go:356] "Generic (PLEG): container finished" podID="8001123a-4ae2-4dc0-93d1-00d7ca27040e" containerID="d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38" exitCode=0 Jul 07 17:33:49.318977 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.318941 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" Jul 07 17:33:49.319171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.318974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" event={"ID":"8001123a-4ae2-4dc0-93d1-00d7ca27040e","Type":"ContainerDied","Data":"d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38"} Jul 07 17:33:49.319171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.319025 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx" event={"ID":"8001123a-4ae2-4dc0-93d1-00d7ca27040e","Type":"ContainerDied","Data":"dad83743448936a541d58db3e9aca08162f7cdd4562389e05ff038430f7659f8"} Jul 07 17:33:49.319171 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.319046 2582 scope.go:117] "RemoveContainer" containerID="d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38" Jul 07 17:33:49.327814 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.327792 2582 scope.go:117] "RemoveContainer" containerID="d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38" Jul 07 17:33:49.328145 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:33:49.328123 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38\": container with ID starting with d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38 not found: ID does not exist" containerID="d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38" Jul 07 17:33:49.328237 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.328151 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38"} err="failed to get container status \"d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38\": rpc error: code = NotFound desc = could not find container \"d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38\": container with ID starting with d4f1e6b1adf6819a945022f6a646557f1471ccdb8e8a52315f2847cc341abe38 not found: ID does not exist" Jul 07 17:33:49.351831 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.351800 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx"] Jul 07 17:33:49.353921 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.353894 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9d5662de-openshift-default-79756f6969-2hqdx"] Jul 07 17:33:49.676009 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:49.675971 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:33:50.934639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:33:50.934608 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8001123a-4ae2-4dc0-93d1-00d7ca27040e" path="/var/lib/kubelet/pods/8001123a-4ae2-4dc0-93d1-00d7ca27040e/volumes" Jul 07 17:34:15.285481 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:34:15.285448 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:34:27.892209 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:34:27.892123 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:35:50.272488 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:35:50.272451 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:35:59.275527 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:35:59.275443 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:36:03.737714 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.737682 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk"] Jul 07 17:36:03.738083 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.738049 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8001123a-4ae2-4dc0-93d1-00d7ca27040e" containerName="istio-proxy" Jul 07 17:36:03.738083 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.738060 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8001123a-4ae2-4dc0-93d1-00d7ca27040e" containerName="istio-proxy" Jul 07 17:36:03.738154 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.738132 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8001123a-4ae2-4dc0-93d1-00d7ca27040e" containerName="istio-proxy" Jul 07 17:36:03.741261 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.741243 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.744405 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.744388 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-88352601-openshift-default-dockercfg-kqnn8\"" Jul 07 17:36:03.770064 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.770036 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk"] Jul 07 17:36:03.902007 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.901974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-credential-socket\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-podinfo\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902044 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istiod-ca-cert\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902109 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzs5d\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-kube-api-access-pzs5d\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902131 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-data\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902318 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902191 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-certs\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902318 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-envoy\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902318 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902258 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-socket\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:03.902318 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:03.902276 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-token\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003105 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-podinfo\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003105 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istiod-ca-cert\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003105 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzs5d\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-kube-api-access-pzs5d\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003345 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-data\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003345 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-certs\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003446 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-envoy\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003501 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-socket\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003501 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-token\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003632 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-credential-socket\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003632 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003605 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-data\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003737 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003639 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-certs\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003851 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003824 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istiod-ca-cert\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003864 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-socket\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.003993 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.003907 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-credential-socket\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.005531 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.005508 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-envoy\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.005639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.005586 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-podinfo\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.012064 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.012037 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-token\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.012174 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.012152 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzs5d\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-kube-api-access-pzs5d\") pod \"e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.051040 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.051010 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:04.195952 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.195925 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk"] Jul 07 17:36:04.197969 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:36:04.197940 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0f1be5_b4b4_4442_955e_5acb1a7dd208.slice/crio-be7d99c2a8161471e547388c49f2b6c3f7322b92ea924dd7974b13153f70154c WatchSource:0}: Error finding container be7d99c2a8161471e547388c49f2b6c3f7322b92ea924dd7974b13153f70154c: Status 404 returned error can't find the container with id be7d99c2a8161471e547388c49f2b6c3f7322b92ea924dd7974b13153f70154c Jul 07 17:36:04.200195 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.200162 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:04.200274 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.200231 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:04.200274 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.200258 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:04.956519 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.956483 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" event={"ID":"0b0f1be5-b4b4-4442-955e-5acb1a7dd208","Type":"ContainerStarted","Data":"377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f"} Jul 07 17:36:04.956997 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.956535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" event={"ID":"0b0f1be5-b4b4-4442-955e-5acb1a7dd208","Type":"ContainerStarted","Data":"be7d99c2a8161471e547388c49f2b6c3f7322b92ea924dd7974b13153f70154c"} Jul 07 17:36:04.980148 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:04.980095 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" podStartSLOduration=1.9800819349999998 podStartE2EDuration="1.980081935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:36:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:36:04.978224263 +0000 UTC m=+1716.838304336" watchObservedRunningTime="2026-07-07 17:36:04.980081935 +0000 UTC m=+1716.840162007" Jul 07 17:36:05.051191 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:05.051157 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:05.056048 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:05.056027 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:05.961731 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:05.961694 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:05.962623 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:05.962603 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:11.663082 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:11.663039 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:36:20.027891 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.027854 2582 generic.go:356] "Generic (PLEG): container finished" podID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerID="377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f" exitCode=0 Jul 07 17:36:20.028303 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.027929 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" event={"ID":"0b0f1be5-b4b4-4442-955e-5acb1a7dd208","Type":"ContainerDied","Data":"377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f"} Jul 07 17:36:20.028359 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.028345 2582 scope.go:117] "RemoveContainer" containerID="377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f" Jul 07 17:36:20.029448 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.029412 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:20.029552 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.029505 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:20.029606 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.029551 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:20.962565 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:20.962518 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:21.039186 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:21.039136 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" event={"ID":"0b0f1be5-b4b4-4442-955e-5acb1a7dd208","Type":"ContainerStarted","Data":"06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e"} Jul 07 17:36:21.051528 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:21.051494 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:21.052683 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:21.052655 2582 patch_prober.go:28] interesting pod/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 17:36:21.052809 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:21.052711 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 17:36:22.052341 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:22.052307 2582 patch_prober.go:28] interesting pod/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 17:36:22.052767 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:22.052358 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 17:36:22.870210 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:22.870177 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:36:23.051446 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:23.051412 2582 patch_prober.go:28] interesting pod/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 17:36:23.051636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:23.051476 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 17:36:24.051837 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:24.051800 2582 patch_prober.go:28] interesting pod/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 17:36:24.052310 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:24.051859 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 17:36:25.055495 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:25.055470 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:25.055948 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:25.055916 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:25.056717 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:25.056697 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:26.903605 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:26.903552 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk"] Jul 07 17:36:27.070226 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:27.070192 2582 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" secret="" err="secret \"e2e-mt-88352601-openshift-default-dockercfg-kqnn8\" not found" Jul 07 17:36:28.076796 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:28.076757 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" containerID="cri-o://06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e" gracePeriod=30 Jul 07 17:36:33.333981 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.333956 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:33.484725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484689 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-credential-socket\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484728 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istiod-ca-cert\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484753 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-envoy\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484797 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-socket\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484824 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-certs\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484906 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-podinfo\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484937 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-token\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.484984 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.484983 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-data\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.485285 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485014 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzs5d\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-kube-api-access-pzs5d\") pod \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\" (UID: \"0b0f1be5-b4b4-4442-955e-5acb1a7dd208\") " Jul 07 17:36:33.485285 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485082 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:36:33.485285 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485215 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:36:33.485285 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485256 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:36:33.485491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485254 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:36:33.485491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485399 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-credential-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.485491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485434 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istiod-ca-cert\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.485491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.485466 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.486106 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.486082 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:36:33.487263 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.487243 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-kube-api-access-pzs5d" (OuterVolumeSpecName: "kube-api-access-pzs5d") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "kube-api-access-pzs5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:36:33.487316 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.487272 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 17:36:33.487500 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.487476 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:36:33.487500 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.487487 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b0f1be5-b4b4-4442-955e-5acb1a7dd208" (UID: "0b0f1be5-b4b4-4442-955e-5acb1a7dd208"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:36:33.586163 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.586133 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-data\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.586163 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.586160 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzs5d\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-kube-api-access-pzs5d\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.586163 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.586172 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-envoy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.586385 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.586180 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-workload-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.586385 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.586189 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-podinfo\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:33.586385 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:33.586197 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b0f1be5-b4b4-4442-955e-5acb1a7dd208-istio-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:36:34.106302 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.106268 2582 generic.go:356] "Generic (PLEG): container finished" podID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerID="06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e" exitCode=0 Jul 07 17:36:34.106463 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.106317 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" Jul 07 17:36:34.106463 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.106356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" event={"ID":"0b0f1be5-b4b4-4442-955e-5acb1a7dd208","Type":"ContainerDied","Data":"06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e"} Jul 07 17:36:34.106463 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.106411 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk" event={"ID":"0b0f1be5-b4b4-4442-955e-5acb1a7dd208","Type":"ContainerDied","Data":"be7d99c2a8161471e547388c49f2b6c3f7322b92ea924dd7974b13153f70154c"} Jul 07 17:36:34.106463 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.106434 2582 scope.go:117] "RemoveContainer" containerID="06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e" Jul 07 17:36:34.115772 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.115755 2582 scope.go:117] "RemoveContainer" containerID="377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f" Jul 07 17:36:34.123708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.123692 2582 scope.go:117] "RemoveContainer" containerID="06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e" Jul 07 17:36:34.123966 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:36:34.123945 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e\": container with ID starting with 06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e not found: ID does not exist" containerID="06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e" Jul 07 17:36:34.124031 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.123974 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e"} err="failed to get container status \"06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e\": rpc error: code = NotFound desc = could not find container \"06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e\": container with ID starting with 06deec9db8870f45ef6670ae08410ae38a509521cbd6089e04ba17abf84c0a9e not found: ID does not exist" Jul 07 17:36:34.124031 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.123990 2582 scope.go:117] "RemoveContainer" containerID="377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f" Jul 07 17:36:34.124196 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:36:34.124181 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f\": container with ID starting with 377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f not found: ID does not exist" containerID="377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f" Jul 07 17:36:34.124239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.124201 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f"} err="failed to get container status \"377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f\": rpc error: code = NotFound desc = could not find container \"377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f\": container with ID starting with 377bef1f50be25af582397823c311fc8c0aba972442672635791d99103fee47f not found: ID does not exist" Jul 07 17:36:34.137185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.137159 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk"] Jul 07 17:36:34.143514 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.143489 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-88352601-openshift-default-7f6bf9d88c-k4vqk"] Jul 07 17:36:34.935024 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:34.934992 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" path="/var/lib/kubelet/pods/0b0f1be5-b4b4-4442-955e-5acb1a7dd208/volumes" Jul 07 17:36:48.405215 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.405177 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f"] Jul 07 17:36:48.405708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.405542 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" Jul 07 17:36:48.405708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.405553 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" Jul 07 17:36:48.405708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.405649 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" Jul 07 17:36:48.418104 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.418076 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.420982 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.420793 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7bdc63cf-openshift-default-dockercfg-lwxxn\"" Jul 07 17:36:48.422424 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.422335 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f"] Jul 07 17:36:48.523940 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.523902 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-credential-socket\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.523946 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-socket\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.523992 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-token\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.524060 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-certs\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524116 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.524078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs4xq\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-kube-api-access-fs4xq\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.524156 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-envoy\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.524185 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-data\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.524201 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istiod-ca-cert\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.524394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.524263 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-podinfo\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625325 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625283 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-socket\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625325 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-token\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625547 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-certs\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625547 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs4xq\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-kube-api-access-fs4xq\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625547 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-envoy\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625752 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625595 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-data\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625752 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istiod-ca-cert\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625752 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625728 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-podinfo\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625928 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-credential-socket\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625928 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625799 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-certs\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625928 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625901 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-data\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.625928 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.625913 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-socket\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.626106 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.626087 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-credential-socket\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.626489 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.626458 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istiod-ca-cert\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.627841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.627815 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-envoy\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.628150 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.628132 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-podinfo\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.636859 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.636838 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs4xq\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-kube-api-access-fs4xq\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.636963 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.636878 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-token\") pod \"e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.729468 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.729376 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:48.872365 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.872334 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f"] Jul 07 17:36:48.874712 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:36:48.874682 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f8e508_4480_4edb_b7bf_01afb9c49a27.slice/crio-b69679814fc589436dceb077b2bd523c177eb95d127b6048002302fc5e8b5901 WatchSource:0}: Error finding container b69679814fc589436dceb077b2bd523c177eb95d127b6048002302fc5e8b5901: Status 404 returned error can't find the container with id b69679814fc589436dceb077b2bd523c177eb95d127b6048002302fc5e8b5901 Jul 07 17:36:48.877070 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.877031 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:48.877149 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.877118 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:48.877197 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:48.877166 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:36:49.180620 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:49.180562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" event={"ID":"b2f8e508-4480-4edb-b7bf-01afb9c49a27","Type":"ContainerStarted","Data":"2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e"} Jul 07 17:36:49.180785 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:49.180638 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" event={"ID":"b2f8e508-4480-4edb-b7bf-01afb9c49a27","Type":"ContainerStarted","Data":"b69679814fc589436dceb077b2bd523c177eb95d127b6048002302fc5e8b5901"} Jul 07 17:36:49.202512 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:49.202453 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podStartSLOduration=1.202433307 podStartE2EDuration="1.202433307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:36:49.201650705 +0000 UTC m=+1761.061730778" watchObservedRunningTime="2026-07-07 17:36:49.202433307 +0000 UTC m=+1761.062513381" Jul 07 17:36:49.729967 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:49.729929 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:49.735073 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:49.735047 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:50.185399 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:50.185359 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:50.186248 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:50.186227 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:36:56.463922 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:36:56.463887 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:37:05.187319 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.187279 2582 patch_prober.go:28] interesting pod/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 17:37:05.187755 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.187345 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 17:37:05.270751 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.270715 2582 generic.go:356] "Generic (PLEG): container finished" podID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerID="2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e" exitCode=0 Jul 07 17:37:05.270916 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.270791 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" event={"ID":"b2f8e508-4480-4edb-b7bf-01afb9c49a27","Type":"ContainerDied","Data":"2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e"} Jul 07 17:37:05.271204 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.271188 2582 scope.go:117] "RemoveContainer" containerID="2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e" Jul 07 17:37:05.272273 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.272247 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:37:05.272330 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.272320 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:37:05.272365 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:05.272356 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:37:06.278493 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:06.278459 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" event={"ID":"b2f8e508-4480-4edb-b7bf-01afb9c49a27","Type":"ContainerStarted","Data":"924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec"} Jul 07 17:37:06.730608 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:06.730553 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:37:06.730864 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:06.730835 2582 patch_prober.go:28] interesting pod/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 17:37:06.730941 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:06.730899 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 17:37:07.730864 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:07.730825 2582 patch_prober.go:28] interesting pod/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 17:37:07.731920 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:07.731454 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 17:37:08.730269 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:08.730220 2582 patch_prober.go:28] interesting pod/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 17:37:08.730487 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:08.730316 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 17:37:09.730169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:09.730128 2582 patch_prober.go:28] interesting pod/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 17:37:09.730792 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:09.730755 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 17:37:10.734077 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:10.734035 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:37:10.734458 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:10.734369 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:37:10.735111 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:10.735091 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:37:11.264450 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:11.264410 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:37:25.069909 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:25.069831 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:37:28.798301 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:28.798275 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:37:28.798301 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:28.798291 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:37:39.978980 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:37:39.978947 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:38:20.374036 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:20.373998 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:38:24.013300 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:24.013263 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f"] Jul 07 17:38:24.013736 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:24.013458 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" containerID="cri-o://924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec" gracePeriod=30 Jul 07 17:38:25.736805 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:25.736764 2582 patch_prober.go:28] interesting pod/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 17:38:25.737284 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:25.736854 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 17:38:29.265391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.265366 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:38:29.348778 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348682 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istiod-ca-cert\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.348778 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348739 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-certs\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348857 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-socket\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348902 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs4xq\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-kube-api-access-fs4xq\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348942 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-envoy\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348970 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-credential-socket\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.348994 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-podinfo\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349017 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:38:29.349365 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349044 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-token\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349365 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349068 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-data\") pod \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\" (UID: \"b2f8e508-4480-4edb-b7bf-01afb9c49a27\") " Jul 07 17:38:29.349365 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349170 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:38:29.349521 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349312 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:38:29.349521 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349479 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istiod-ca-cert\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.349521 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349501 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.349820 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.349545 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:38:29.350218 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.350192 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:38:29.351278 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.351238 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:38:29.351416 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.351357 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-kube-api-access-fs4xq" (OuterVolumeSpecName: "kube-api-access-fs4xq") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "kube-api-access-fs4xq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:38:29.351612 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.351569 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 17:38:29.351712 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.351627 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2f8e508-4480-4edb-b7bf-01afb9c49a27" (UID: "b2f8e508-4480-4edb-b7bf-01afb9c49a27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:38:29.449909 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449879 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-workload-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.449909 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449903 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs4xq\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-kube-api-access-fs4xq\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.449909 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449914 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-envoy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.450131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449922 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-credential-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.450131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449950 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-podinfo\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.450131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449959 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.450131 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.449966 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2f8e508-4480-4edb-b7bf-01afb9c49a27-istio-data\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:38:29.683597 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.683541 2582 generic.go:356] "Generic (PLEG): container finished" podID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerID="924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec" exitCode=0 Jul 07 17:38:29.683775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.683612 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" Jul 07 17:38:29.683775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.683615 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" event={"ID":"b2f8e508-4480-4edb-b7bf-01afb9c49a27","Type":"ContainerDied","Data":"924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec"} Jul 07 17:38:29.683775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.683719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f" event={"ID":"b2f8e508-4480-4edb-b7bf-01afb9c49a27","Type":"ContainerDied","Data":"b69679814fc589436dceb077b2bd523c177eb95d127b6048002302fc5e8b5901"} Jul 07 17:38:29.683775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.683740 2582 scope.go:117] "RemoveContainer" containerID="924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec" Jul 07 17:38:29.693545 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.693529 2582 scope.go:117] "RemoveContainer" containerID="2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e" Jul 07 17:38:29.702235 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.702219 2582 scope.go:117] "RemoveContainer" containerID="924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec" Jul 07 17:38:29.702487 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:38:29.702465 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec\": container with ID starting with 924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec not found: ID does not exist" containerID="924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec" Jul 07 17:38:29.702540 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.702498 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec"} err="failed to get container status \"924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec\": rpc error: code = NotFound desc = could not find container \"924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec\": container with ID starting with 924238bbcfb0a0adab5ca3e526a4011df623a88db71a7fd70f0c001be0a49cec not found: ID does not exist" Jul 07 17:38:29.702540 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.702515 2582 scope.go:117] "RemoveContainer" containerID="2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e" Jul 07 17:38:29.703053 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:38:29.703034 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e\": container with ID starting with 2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e not found: ID does not exist" containerID="2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e" Jul 07 17:38:29.703100 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.703060 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e"} err="failed to get container status \"2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e\": rpc error: code = NotFound desc = could not find container \"2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e\": container with ID starting with 2eb7683e742a74f68160210b526a4081eec578041e7e1c90b0e7f0cb2a59517e not found: ID does not exist" Jul 07 17:38:29.707493 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.707472 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f"] Jul 07 17:38:29.712199 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.712178 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7bdc63cf-openshift-default-845647954c-57q9f"] Jul 07 17:38:29.972147 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:29.972059 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:38:30.934481 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:30.934447 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" path="/var/lib/kubelet/pods/b2f8e508-4480-4edb-b7bf-01afb9c49a27/volumes" Jul 07 17:38:43.368847 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:38:43.368808 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:39:29.216724 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.216694 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc"] Jul 07 17:39:29.217123 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217099 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" Jul 07 17:39:29.217123 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217112 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" Jul 07 17:39:29.217123 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217124 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" Jul 07 17:39:29.217239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217129 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" Jul 07 17:39:29.217239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217151 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" Jul 07 17:39:29.217239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217159 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" Jul 07 17:39:29.217239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217225 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0f1be5-b4b4-4442-955e-5acb1a7dd208" containerName="istio-proxy" Jul 07 17:39:29.217239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217232 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" Jul 07 17:39:29.217239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.217238 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2f8e508-4480-4edb-b7bf-01afb9c49a27" containerName="istio-proxy" Jul 07 17:39:29.221655 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.221636 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.226058 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.226035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-567860-openshift-default-dockercfg-9cqwm\"" Jul 07 17:39:29.242590 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.242552 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc"] Jul 07 17:39:29.289001 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.288965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-certs\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289001 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.288999 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0c964a-a23d-470d-a787-a034453d8db9-istiod-ca-cert\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289030 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-credential-socket\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0c964a-a23d-470d-a787-a034453d8db9-istio-podinfo\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-data\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289198 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-socket\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-istio-token\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcds2\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-kube-api-access-dcds2\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.289448 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.289286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-envoy\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.389967 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.389929 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0c964a-a23d-470d-a787-a034453d8db9-istio-podinfo\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.389967 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.389967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-data\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390006 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-socket\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-istio-token\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcds2\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-kube-api-access-dcds2\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390104 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-envoy\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390181 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-certs\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0c964a-a23d-470d-a787-a034453d8db9-istiod-ca-cert\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390540 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-credential-socket\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390540 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390469 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-socket\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390675 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390548 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-credential-socket\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390675 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390590 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-certs\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390749 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390679 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-data\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.390971 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.390952 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0c964a-a23d-470d-a787-a034453d8db9-istiod-ca-cert\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.392406 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.392380 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-envoy\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.392532 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.392415 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0c964a-a23d-470d-a787-a034453d8db9-istio-podinfo\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.407807 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.407786 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-istio-token\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.409991 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.409972 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcds2\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-kube-api-access-dcds2\") pod \"e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.532333 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.532237 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:29.682070 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.682039 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc"] Jul 07 17:39:29.685492 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:39:29.685459 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0c964a_a23d_470d_a787_a034453d8db9.slice/crio-2d5af0002215872c312d77987a4fa3238a5c3a39cc9c58717467ceaf262566a4 WatchSource:0}: Error finding container 2d5af0002215872c312d77987a4fa3238a5c3a39cc9c58717467ceaf262566a4: Status 404 returned error can't find the container with id 2d5af0002215872c312d77987a4fa3238a5c3a39cc9c58717467ceaf262566a4 Jul 07 17:39:29.687520 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.687486 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:39:29.687607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.687552 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:39:29.687607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.687595 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:39:29.968243 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.968197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" event={"ID":"bd0c964a-a23d-470d-a787-a034453d8db9","Type":"ContainerStarted","Data":"4965d902c921e530811bcf45e7a8ca2d61152894d3c8e5e0631cdd4ae37b4764"} Jul 07 17:39:29.968407 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:29.968256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" event={"ID":"bd0c964a-a23d-470d-a787-a034453d8db9","Type":"ContainerStarted","Data":"2d5af0002215872c312d77987a4fa3238a5c3a39cc9c58717467ceaf262566a4"} Jul 07 17:39:30.001214 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:30.001167 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podStartSLOduration=1.001151829 podStartE2EDuration="1.001151829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:39:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:39:29.998926709 +0000 UTC m=+1921.859006781" watchObservedRunningTime="2026-07-07 17:39:30.001151829 +0000 UTC m=+1921.861231879" Jul 07 17:39:30.532823 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:30.532791 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:30.537824 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:30.537800 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:30.972544 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:30.972514 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:30.973441 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:30.973421 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:31.280880 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:31.280800 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:39:45.576263 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:45.576220 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:39:47.013759 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:47.013718 2582 patch_prober.go:28] interesting pod/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 17:39:47.014147 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:47.013798 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 17:39:48.064368 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:48.064334 2582 generic.go:356] "Generic (PLEG): container finished" podID="bd0c964a-a23d-470d-a787-a034453d8db9" containerID="4965d902c921e530811bcf45e7a8ca2d61152894d3c8e5e0631cdd4ae37b4764" exitCode=0 Jul 07 17:39:48.064820 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:48.064402 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" event={"ID":"bd0c964a-a23d-470d-a787-a034453d8db9","Type":"ContainerDied","Data":"4965d902c921e530811bcf45e7a8ca2d61152894d3c8e5e0631cdd4ae37b4764"} Jul 07 17:39:48.064897 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:48.064879 2582 scope.go:117] "RemoveContainer" containerID="4965d902c921e530811bcf45e7a8ca2d61152894d3c8e5e0631cdd4ae37b4764" Jul 07 17:39:48.066024 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:48.065996 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:39:48.066103 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:48.066090 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:39:48.066143 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:48.066127 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:39:49.070639 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:49.070601 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" event={"ID":"bd0c964a-a23d-470d-a787-a034453d8db9","Type":"ContainerStarted","Data":"26131ac3a1182d0b583a6e2494b5df10bc6a49c8049ac7e0346b8db803bc0fdf"} Jul 07 17:39:49.533095 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:49.533059 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:49.533421 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:49.533157 2582 patch_prober.go:28] interesting pod/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 17:39:49.533654 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:49.533625 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 17:39:50.532737 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:50.532695 2582 patch_prober.go:28] interesting pod/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 17:39:50.533356 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:50.533320 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 17:39:51.533666 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:51.533615 2582 patch_prober.go:28] interesting pod/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 17:39:51.534199 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:51.533689 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 17:39:52.533350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:52.533297 2582 patch_prober.go:28] interesting pod/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 17:39:52.533607 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:52.533376 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 17:39:53.536896 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:53.536864 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:53.537343 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:53.537238 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:53.538007 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:53.537985 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:39:57.172766 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:39:57.172730 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:40:08.965698 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:40:08.965665 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:40:24.471692 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:40:24.471602 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:40:41.767730 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:40:41.767692 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:41:19.067510 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:19.067475 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:41:22.408108 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:22.408075 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc"] Jul 07 17:41:22.408554 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:22.408302 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" containerID="cri-o://26131ac3a1182d0b583a6e2494b5df10bc6a49c8049ac7e0346b8db803bc0fdf" gracePeriod=30 Jul 07 17:41:23.539861 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:23.539813 2582 patch_prober.go:28] interesting pod/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 17:41:23.540268 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:23.539886 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 17:41:27.543666 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.543633 2582 generic.go:356] "Generic (PLEG): container finished" podID="bd0c964a-a23d-470d-a787-a034453d8db9" containerID="26131ac3a1182d0b583a6e2494b5df10bc6a49c8049ac7e0346b8db803bc0fdf" exitCode=0 Jul 07 17:41:27.544167 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.543691 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" event={"ID":"bd0c964a-a23d-470d-a787-a034453d8db9","Type":"ContainerDied","Data":"26131ac3a1182d0b583a6e2494b5df10bc6a49c8049ac7e0346b8db803bc0fdf"} Jul 07 17:41:27.544167 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.543740 2582 scope.go:117] "RemoveContainer" containerID="4965d902c921e530811bcf45e7a8ca2d61152894d3c8e5e0631cdd4ae37b4764" Jul 07 17:41:27.663797 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.663774 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:41:27.672225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672160 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-istio-token\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672225 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672195 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcds2\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-kube-api-access-dcds2\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672415 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672226 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-credential-socket\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672415 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672253 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-socket\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672415 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672326 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-data\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672415 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672381 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-certs\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672415 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672405 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0c964a-a23d-470d-a787-a034453d8db9-istiod-ca-cert\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672716 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672462 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-envoy\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672716 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672510 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:41:27.672716 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672522 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0c964a-a23d-470d-a787-a034453d8db9-istio-podinfo\") pod \"bd0c964a-a23d-470d-a787-a034453d8db9\" (UID: \"bd0c964a-a23d-470d-a787-a034453d8db9\") " Jul 07 17:41:27.672716 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672561 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:41:27.672716 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672695 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:41:27.672992 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672800 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0c964a-a23d-470d-a787-a034453d8db9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:41:27.672992 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672928 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-credential-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.672992 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672947 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.672992 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672961 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-workload-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.672992 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.672975 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0c964a-a23d-470d-a787-a034453d8db9-istiod-ca-cert\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.673259 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.673233 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:41:27.674612 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.674554 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd0c964a-a23d-470d-a787-a034453d8db9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 17:41:27.674726 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.674702 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:41:27.674908 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.674887 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-kube-api-access-dcds2" (OuterVolumeSpecName: "kube-api-access-dcds2") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "kube-api-access-dcds2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:41:27.675050 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.674970 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd0c964a-a23d-470d-a787-a034453d8db9" (UID: "bd0c964a-a23d-470d-a787-a034453d8db9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:41:27.773510 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.773476 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-data\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.773510 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.773503 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0c964a-a23d-470d-a787-a034453d8db9-istio-envoy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.773510 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.773512 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0c964a-a23d-470d-a787-a034453d8db9-istio-podinfo\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.773768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.773524 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-istio-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:27.773768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:27.773539 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcds2\" (UniqueName: \"kubernetes.io/projected/bd0c964a-a23d-470d-a787-a034453d8db9-kube-api-access-dcds2\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:41:28.549018 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:28.548985 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" Jul 07 17:41:28.549467 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:28.549025 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc" event={"ID":"bd0c964a-a23d-470d-a787-a034453d8db9","Type":"ContainerDied","Data":"2d5af0002215872c312d77987a4fa3238a5c3a39cc9c58717467ceaf262566a4"} Jul 07 17:41:28.549467 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:28.549081 2582 scope.go:117] "RemoveContainer" containerID="26131ac3a1182d0b583a6e2494b5df10bc6a49c8049ac7e0346b8db803bc0fdf" Jul 07 17:41:28.573861 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:28.573829 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc"] Jul 07 17:41:28.578298 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:28.578273 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-567860-openshift-default-c9b8b677f-tm7xc"] Jul 07 17:41:28.934999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:28.934967 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" path="/var/lib/kubelet/pods/bd0c964a-a23d-470d-a787-a034453d8db9/volumes" Jul 07 17:41:31.812310 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812276 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9"] Jul 07 17:41:31.812796 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812776 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" Jul 07 17:41:31.812857 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812800 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" Jul 07 17:41:31.812857 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812825 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" Jul 07 17:41:31.812857 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812834 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" Jul 07 17:41:31.813003 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812916 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" Jul 07 17:41:31.813003 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.812942 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0c964a-a23d-470d-a787-a034453d8db9" containerName="istio-proxy" Jul 07 17:41:31.817684 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.817663 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.821077 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.821050 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4c45e1-openshift-default-dockercfg-cm24p\"" Jul 07 17:41:31.839339 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.839315 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9"] Jul 07 17:41:31.910551 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910522 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-socket\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.910551 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910553 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-certs\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.910856 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910593 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-data\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.910856 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-istio-token\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.910856 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d670d734-3098-478b-aef2-3a9495b5e396-istiod-ca-cert\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.910856 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-credential-socket\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.910856 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d670d734-3098-478b-aef2-3a9495b5e396-istio-podinfo\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.911044 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-envoy\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:31.911044 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:31.910920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szd8w\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-kube-api-access-szd8w\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.011899 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.011862 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-credential-socket\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.011899 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.011899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d670d734-3098-478b-aef2-3a9495b5e396-istio-podinfo\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-envoy\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szd8w\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-kube-api-access-szd8w\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012287 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-socket\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012287 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012213 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-certs\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012294 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-credential-socket\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012448 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012412 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-data\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012505 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-istio-token\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012505 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012481 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-certs\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d670d734-3098-478b-aef2-3a9495b5e396-istiod-ca-cert\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012636 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012602 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-socket\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.012787 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.012768 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-data\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.013090 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.013068 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d670d734-3098-478b-aef2-3a9495b5e396-istiod-ca-cert\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.014802 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.014779 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-envoy\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.015027 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.015005 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d670d734-3098-478b-aef2-3a9495b5e396-istio-podinfo\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.021947 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.021922 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-istio-token\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.022048 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.022035 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szd8w\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-kube-api-access-szd8w\") pod \"e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.128687 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.128656 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:32.272394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.272369 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9"] Jul 07 17:41:32.274145 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:41:32.274091 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd670d734_3098_478b_aef2_3a9495b5e396.slice/crio-f45779e28bd4b95014d4765b73e71afbd1806390cd25e51bc1295421553af7e4 WatchSource:0}: Error finding container f45779e28bd4b95014d4765b73e71afbd1806390cd25e51bc1295421553af7e4: Status 404 returned error can't find the container with id f45779e28bd4b95014d4765b73e71afbd1806390cd25e51bc1295421553af7e4 Jul 07 17:41:32.276487 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.276451 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:41:32.276603 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.276536 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:41:32.276664 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.276628 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:41:32.568052 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.567958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" event={"ID":"d670d734-3098-478b-aef2-3a9495b5e396","Type":"ContainerStarted","Data":"4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0"} Jul 07 17:41:32.568052 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.568011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" event={"ID":"d670d734-3098-478b-aef2-3a9495b5e396","Type":"ContainerStarted","Data":"f45779e28bd4b95014d4765b73e71afbd1806390cd25e51bc1295421553af7e4"} Jul 07 17:41:32.595388 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:32.595336 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" podStartSLOduration=1.595320608 podStartE2EDuration="1.595320608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:32.592773369 +0000 UTC m=+2044.452853480" watchObservedRunningTime="2026-07-07 17:41:32.595320608 +0000 UTC m=+2044.455400679" Jul 07 17:41:33.129775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:33.129734 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:33.134830 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:33.134805 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:33.573846 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:33.573767 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:33.574969 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:33.574946 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:34.690499 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:34.690463 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:41:51.978729 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:51.978688 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:41:52.673770 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:52.673733 2582 generic.go:356] "Generic (PLEG): container finished" podID="d670d734-3098-478b-aef2-3a9495b5e396" containerID="4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0" exitCode=0 Jul 07 17:41:52.673987 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:52.673805 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" event={"ID":"d670d734-3098-478b-aef2-3a9495b5e396","Type":"ContainerDied","Data":"4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0"} Jul 07 17:41:52.674239 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:52.674224 2582 scope.go:117] "RemoveContainer" containerID="4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0" Jul 07 17:41:52.675350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:52.675320 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:41:52.675443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:52.675408 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:41:52.675497 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:52.675446 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:41:53.680816 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:53.680783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" event={"ID":"d670d734-3098-478b-aef2-3a9495b5e396","Type":"ContainerStarted","Data":"0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1"} Jul 07 17:41:54.130106 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:54.130067 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:54.130530 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:54.130441 2582 patch_prober.go:28] interesting pod/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 17:41:54.131285 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:54.131253 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 17:41:55.129293 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:55.129253 2582 patch_prober.go:28] interesting pod/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 17:41:55.129952 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:55.129916 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 17:41:56.130016 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:56.129975 2582 patch_prober.go:28] interesting pod/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 17:41:56.130662 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:56.130627 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 17:41:57.129990 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:57.129947 2582 patch_prober.go:28] interesting pod/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 17:41:57.130305 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:57.130272 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 17:41:58.133298 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:58.133268 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:58.133723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:58.133512 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:41:58.134263 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:41:58.134244 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:42:02.842248 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:02.842216 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9"] Jul 07 17:42:02.842689 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:02.842424 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" containerID="cri-o://0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1" gracePeriod=30 Jul 07 17:42:05.973081 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:05.973040 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:42:08.107095 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.107019 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:42:08.169399 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169362 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szd8w\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-kube-api-access-szd8w\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169614 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169426 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-envoy\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169614 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169474 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d670d734-3098-478b-aef2-3a9495b5e396-istiod-ca-cert\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169614 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169514 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d670d734-3098-478b-aef2-3a9495b5e396-istio-podinfo\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169614 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169589 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-data\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169620 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-socket\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169650 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-certs\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169683 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-credential-socket\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.169841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169714 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-istio-token\") pod \"d670d734-3098-478b-aef2-3a9495b5e396\" (UID: \"d670d734-3098-478b-aef2-3a9495b5e396\") " Jul 07 17:42:08.170058 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.169960 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d670d734-3098-478b-aef2-3a9495b5e396-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:42:08.170123 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.170026 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:42:08.170301 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.170266 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:42:08.170419 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.170339 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:42:08.170551 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.170519 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-data" (OuterVolumeSpecName: "istio-data") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:42:08.171726 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.171683 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:42:08.171847 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.171730 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d670d734-3098-478b-aef2-3a9495b5e396-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 17:42:08.171847 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.171754 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-kube-api-access-szd8w" (OuterVolumeSpecName: "kube-api-access-szd8w") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "kube-api-access-szd8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:42:08.171977 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.171960 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-istio-token" (OuterVolumeSpecName: "istio-token") pod "d670d734-3098-478b-aef2-3a9495b5e396" (UID: "d670d734-3098-478b-aef2-3a9495b5e396"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:42:08.270999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.270963 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-data\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.270999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.270996 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.270999 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271004 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-workload-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.271258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271013 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-credential-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.271258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271021 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-istio-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.271258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271030 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szd8w\" (UniqueName: \"kubernetes.io/projected/d670d734-3098-478b-aef2-3a9495b5e396-kube-api-access-szd8w\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.271258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271039 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d670d734-3098-478b-aef2-3a9495b5e396-istio-envoy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.271258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271047 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d670d734-3098-478b-aef2-3a9495b5e396-istiod-ca-cert\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.271258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.271058 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d670d734-3098-478b-aef2-3a9495b5e396-istio-podinfo\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:42:08.756090 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.756058 2582 generic.go:356] "Generic (PLEG): container finished" podID="d670d734-3098-478b-aef2-3a9495b5e396" containerID="0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1" exitCode=0 Jul 07 17:42:08.756272 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.756101 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" event={"ID":"d670d734-3098-478b-aef2-3a9495b5e396","Type":"ContainerDied","Data":"0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1"} Jul 07 17:42:08.756272 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.756107 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" Jul 07 17:42:08.756272 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.756143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9" event={"ID":"d670d734-3098-478b-aef2-3a9495b5e396","Type":"ContainerDied","Data":"f45779e28bd4b95014d4765b73e71afbd1806390cd25e51bc1295421553af7e4"} Jul 07 17:42:08.756272 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.756167 2582 scope.go:117] "RemoveContainer" containerID="0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1" Jul 07 17:42:08.765959 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.765941 2582 scope.go:117] "RemoveContainer" containerID="4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0" Jul 07 17:42:08.773470 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.773454 2582 scope.go:117] "RemoveContainer" containerID="0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1" Jul 07 17:42:08.773775 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:42:08.773753 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1\": container with ID starting with 0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1 not found: ID does not exist" containerID="0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1" Jul 07 17:42:08.773846 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.773783 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1"} err="failed to get container status \"0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1\": rpc error: code = NotFound desc = could not find container \"0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1\": container with ID starting with 0061b3a323c559db57eded04ff16c16abd903a5ea42b5fdaba2d269dfbf2eeb1 not found: ID does not exist" Jul 07 17:42:08.773846 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.773800 2582 scope.go:117] "RemoveContainer" containerID="4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0" Jul 07 17:42:08.774033 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:42:08.774015 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0\": container with ID starting with 4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0 not found: ID does not exist" containerID="4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0" Jul 07 17:42:08.774079 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.774038 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0"} err="failed to get container status \"4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0\": rpc error: code = NotFound desc = could not find container \"4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0\": container with ID starting with 4b6b099b1dc9b78f9552d6ef6daa101733cdf63e2ea7de32b103d7423fd77ad0 not found: ID does not exist" Jul 07 17:42:08.779637 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.779613 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9"] Jul 07 17:42:08.787214 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.787193 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4c45e1-openshift-default-7d49798689-fhkt9"] Jul 07 17:42:08.935022 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:08.934992 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d670d734-3098-478b-aef2-3a9495b5e396" path="/var/lib/kubelet/pods/d670d734-3098-478b-aef2-3a9495b5e396/volumes" Jul 07 17:42:12.047396 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.047362 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5"] Jul 07 17:42:12.047870 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.047849 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" Jul 07 17:42:12.047915 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.047874 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" Jul 07 17:42:12.047915 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.047892 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" Jul 07 17:42:12.047915 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.047901 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" Jul 07 17:42:12.048032 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.048019 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" Jul 07 17:42:12.052722 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.052699 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.056039 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.056018 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ce87c9-openshift-default-dockercfg-sxjm5\"" Jul 07 17:42:12.069887 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.069855 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5"] Jul 07 17:42:12.101393 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101358 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-data\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101559 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istiod-ca-cert\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101559 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101482 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-socket\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101559 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-credential-socket\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101559 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101530 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-token\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101559 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-podinfo\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101763 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101606 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p59fj\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-kube-api-access-p59fj\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101763 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-envoy\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.101763 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.101657 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-certs\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203011 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.202973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-data\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203011 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istiod-ca-cert\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203053 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-socket\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203071 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-credential-socket\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-token\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203108 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-podinfo\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p59fj\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-kube-api-access-p59fj\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-envoy\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203254 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203195 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-certs\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203610 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203420 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-data\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203610 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203551 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-credential-socket\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203692 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203612 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-socket\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.203745 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203709 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-certs\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.204001 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.203979 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istiod-ca-cert\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.205475 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.205452 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-envoy\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.205641 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.205622 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-podinfo\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.212462 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.212433 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-token\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.213890 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.213866 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p59fj\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-kube-api-access-p59fj\") pod \"e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.363281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.363255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:12.367914 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.367891 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:42:12.513870 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.513845 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5"] Jul 07 17:42:12.515540 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:42:12.515510 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3230a74_3332_4fd3_a8db_68e2bd1865ff.slice/crio-a22227312fa65d79930fb58f7697b0fdcf8ddb36212ac71e21e7f6c6c30fa39e WatchSource:0}: Error finding container a22227312fa65d79930fb58f7697b0fdcf8ddb36212ac71e21e7f6c6c30fa39e: Status 404 returned error can't find the container with id a22227312fa65d79930fb58f7697b0fdcf8ddb36212ac71e21e7f6c6c30fa39e Jul 07 17:42:12.517758 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.517721 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:42:12.517873 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.517801 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:42:12.517873 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.517829 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:42:12.777964 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.777884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" event={"ID":"d3230a74-3332-4fd3-a8db-68e2bd1865ff","Type":"ContainerStarted","Data":"80cb44d66bc2a3d26d7f598e97d6057b5b6a6d2abfa065d70065f4740cf1b582"} Jul 07 17:42:12.777964 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.777937 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" event={"ID":"d3230a74-3332-4fd3-a8db-68e2bd1865ff","Type":"ContainerStarted","Data":"a22227312fa65d79930fb58f7697b0fdcf8ddb36212ac71e21e7f6c6c30fa39e"} Jul 07 17:42:12.806350 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:12.806290 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podStartSLOduration=0.806273565 podStartE2EDuration="806.273565ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:12.805165676 +0000 UTC m=+2084.665245750" watchObservedRunningTime="2026-07-07 17:42:12.806273565 +0000 UTC m=+2084.666353713" Jul 07 17:42:13.364060 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:13.364028 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:13.369118 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:13.369096 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:13.783488 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:13.783408 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:13.784538 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:13.784515 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:21.979513 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:21.979477 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:42:28.831912 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:28.831884 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:42:28.832398 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:28.832081 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:42:29.824773 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:29.824728 2582 patch_prober.go:28] interesting pod/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 17:42:29.824953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:29.824817 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 17:42:30.869780 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:30.869747 2582 generic.go:356] "Generic (PLEG): container finished" podID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerID="80cb44d66bc2a3d26d7f598e97d6057b5b6a6d2abfa065d70065f4740cf1b582" exitCode=0 Jul 07 17:42:30.870217 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:30.869820 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" event={"ID":"d3230a74-3332-4fd3-a8db-68e2bd1865ff","Type":"ContainerDied","Data":"80cb44d66bc2a3d26d7f598e97d6057b5b6a6d2abfa065d70065f4740cf1b582"} Jul 07 17:42:30.870263 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:30.870229 2582 scope.go:117] "RemoveContainer" containerID="80cb44d66bc2a3d26d7f598e97d6057b5b6a6d2abfa065d70065f4740cf1b582" Jul 07 17:42:30.871279 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:30.871251 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:42:30.871343 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:30.871329 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:42:30.871379 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:30.871362 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 17:42:31.876569 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:31.876528 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" event={"ID":"d3230a74-3332-4fd3-a8db-68e2bd1865ff","Type":"ContainerStarted","Data":"93f06e409c4f06a5c02adfa91f2b15bb93f7fe9ce9e57e1e3dd14d56cb5cc1af"} Jul 07 17:42:32.364438 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:32.364406 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:32.364649 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:32.364626 2582 patch_prober.go:28] interesting pod/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 17:42:32.364708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:32.364686 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 17:42:33.364613 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:33.364561 2582 patch_prober.go:28] interesting pod/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 17:42:33.365010 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:33.364638 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 17:42:34.364422 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:34.364385 2582 patch_prober.go:28] interesting pod/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 17:42:34.364604 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:34.364446 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 17:42:35.364352 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:35.364307 2582 patch_prober.go:28] interesting pod/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 17:42:35.364893 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:35.364372 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 17:42:36.367529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:36.367497 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:36.368144 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:36.367967 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:36.368776 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:36.368758 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:42:36.577557 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:42:36.577524 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:43:03.675675 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:43:03.675634 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:43:18.576372 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:43:18.576338 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:43:32.376264 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:43:32.376178 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:43:51.966035 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:43:51.965997 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:44:11.481910 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:44:11.481876 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:44:28.472352 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:44:28.472311 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:44:46.364910 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:44:46.364876 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:45:00.191042 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.190959 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724105-8mmq5"] Jul 07 17:45:00.191519 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.191445 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d670d734-3098-478b-aef2-3a9495b5e396" containerName="istio-proxy" Jul 07 17:45:00.194666 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.194650 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:00.201732 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.201705 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp"] Jul 07 17:45:00.204776 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.204760 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wbb9r\"" Jul 07 17:45:00.205767 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.205753 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:00.217704 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.217680 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8"] Jul 07 17:45:00.221067 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.221052 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:00.235279 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.235244 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724105-8mmq5"] Jul 07 17:45:00.240455 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.240430 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp"] Jul 07 17:45:00.242786 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.242765 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8"] Jul 07 17:45:00.246316 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.246290 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dfj\" (UniqueName: \"kubernetes.io/projected/28c4e8b0-dc1c-4d2a-b607-a287acab972a-kube-api-access-h2dfj\") pod \"maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8\" (UID: \"28c4e8b0-dc1c-4d2a-b607-a287acab972a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:00.246529 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.246507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx7bb\" (UniqueName: \"kubernetes.io/projected/43b880e6-527e-4d01-8adb-023d41bb7517-kube-api-access-gx7bb\") pod \"maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp\" (UID: \"43b880e6-527e-4d01-8adb-023d41bb7517\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:00.246695 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.246670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hbc\" (UniqueName: \"kubernetes.io/projected/b7d928e9-4877-422a-833b-41685e722ba3-kube-api-access-n2hbc\") pod \"maas-api-key-cleanup-29724105-8mmq5\" (UID: \"b7d928e9-4877-422a-833b-41685e722ba3\") " pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:00.347471 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.347433 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7bb\" (UniqueName: \"kubernetes.io/projected/43b880e6-527e-4d01-8adb-023d41bb7517-kube-api-access-gx7bb\") pod \"maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp\" (UID: \"43b880e6-527e-4d01-8adb-023d41bb7517\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:00.347471 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.347483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hbc\" (UniqueName: \"kubernetes.io/projected/b7d928e9-4877-422a-833b-41685e722ba3-kube-api-access-n2hbc\") pod \"maas-api-key-cleanup-29724105-8mmq5\" (UID: \"b7d928e9-4877-422a-833b-41685e722ba3\") " pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:00.347725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.347567 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dfj\" (UniqueName: \"kubernetes.io/projected/28c4e8b0-dc1c-4d2a-b607-a287acab972a-kube-api-access-h2dfj\") pod \"maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8\" (UID: \"28c4e8b0-dc1c-4d2a-b607-a287acab972a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:00.356616 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.356569 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx7bb\" (UniqueName: \"kubernetes.io/projected/43b880e6-527e-4d01-8adb-023d41bb7517-kube-api-access-gx7bb\") pod \"maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp\" (UID: \"43b880e6-527e-4d01-8adb-023d41bb7517\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:00.356767 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.356661 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hbc\" (UniqueName: \"kubernetes.io/projected/b7d928e9-4877-422a-833b-41685e722ba3-kube-api-access-n2hbc\") pod \"maas-api-key-cleanup-29724105-8mmq5\" (UID: \"b7d928e9-4877-422a-833b-41685e722ba3\") " pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:00.356811 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.356763 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dfj\" (UniqueName: \"kubernetes.io/projected/28c4e8b0-dc1c-4d2a-b607-a287acab972a-kube-api-access-h2dfj\") pod \"maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8\" (UID: \"28c4e8b0-dc1c-4d2a-b607-a287acab972a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:00.506444 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.506327 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:00.514615 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.514584 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:00.531091 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.531061 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:00.674062 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.673909 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724105-8mmq5"] Jul 07 17:45:00.678160 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:45:00.677768 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d928e9_4877_422a_833b_41685e722ba3.slice/crio-586554b38f8cc14deaad91d948a8f5b3ad2caa05999a6b2bbe974d3a0dec37f1 WatchSource:0}: Error finding container 586554b38f8cc14deaad91d948a8f5b3ad2caa05999a6b2bbe974d3a0dec37f1: Status 404 returned error can't find the container with id 586554b38f8cc14deaad91d948a8f5b3ad2caa05999a6b2bbe974d3a0dec37f1 Jul 07 17:45:00.680064 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.680042 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:45:00.901329 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.901295 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp"] Jul 07 17:45:00.904168 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:45:00.904137 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b880e6_527e_4d01_8adb_023d41bb7517.slice/crio-81ff3575490ed2140defae9efc199bbc8adcaaa45596d83d35e326a622788514 WatchSource:0}: Error finding container 81ff3575490ed2140defae9efc199bbc8adcaaa45596d83d35e326a622788514: Status 404 returned error can't find the container with id 81ff3575490ed2140defae9efc199bbc8adcaaa45596d83d35e326a622788514 Jul 07 17:45:00.930183 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:45:00.930158 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c4e8b0_dc1c_4d2a_b607_a287acab972a.slice/crio-b9e9bd376f93b168ba1a4cf279e2d9e16976413344609640f391d9019103202e WatchSource:0}: Error finding container b9e9bd376f93b168ba1a4cf279e2d9e16976413344609640f391d9019103202e: Status 404 returned error can't find the container with id b9e9bd376f93b168ba1a4cf279e2d9e16976413344609640f391d9019103202e Jul 07 17:45:00.934712 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:00.934691 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8"] Jul 07 17:45:01.610170 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:01.610130 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" event={"ID":"b7d928e9-4877-422a-833b-41685e722ba3","Type":"ContainerStarted","Data":"586554b38f8cc14deaad91d948a8f5b3ad2caa05999a6b2bbe974d3a0dec37f1"} Jul 07 17:45:01.612779 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:01.612745 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" event={"ID":"43b880e6-527e-4d01-8adb-023d41bb7517","Type":"ContainerStarted","Data":"81ff3575490ed2140defae9efc199bbc8adcaaa45596d83d35e326a622788514"} Jul 07 17:45:01.614635 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:01.614598 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" event={"ID":"28c4e8b0-dc1c-4d2a-b607-a287acab972a","Type":"ContainerStarted","Data":"b9e9bd376f93b168ba1a4cf279e2d9e16976413344609640f391d9019103202e"} Jul 07 17:45:03.626621 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:03.626585 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" event={"ID":"b7d928e9-4877-422a-833b-41685e722ba3","Type":"ContainerStarted","Data":"73fb01ee3598204ac94ff7c93e6e436a7fd39754bf538d11f1fa526131df45e7"} Jul 07 17:45:03.628016 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:03.627991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" event={"ID":"43b880e6-527e-4d01-8adb-023d41bb7517","Type":"ContainerStarted","Data":"c84f2c99b94904a867c543f1cc86f97889f86fc4f2153418b05bc73ae2f9f81f"} Jul 07 17:45:03.629416 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:03.629391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" event={"ID":"28c4e8b0-dc1c-4d2a-b607-a287acab972a","Type":"ContainerStarted","Data":"1012b5a310191114e8ec89a33da9c6b28393b4beec70af4ee489a51095859bea"} Jul 07 17:45:03.645906 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:03.645861 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" podStartSLOduration=1.206866469 podStartE2EDuration="3.645849236s" totalImagesPullingTime="2.438982767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:45:03.644125857 +0000 UTC m=+2255.504205943" watchObservedRunningTime="2026-07-07 17:45:03.645849236 +0000 UTC m=+2255.505929308" Jul 07 17:45:03.662065 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:03.662022 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" podStartSLOduration=1.4799468789999999 podStartE2EDuration="3.662009773s" totalImagesPullingTime="2.182062894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:45:03.660377878 +0000 UTC m=+2255.520457964" watchObservedRunningTime="2026-07-07 17:45:03.662009773 +0000 UTC m=+2255.522089844" Jul 07 17:45:03.679126 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:03.679080 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" podStartSLOduration=1.5268104070000001 podStartE2EDuration="3.679065362s" totalImagesPullingTime="2.152254955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:45:03.677732994 +0000 UTC m=+2255.537813065" watchObservedRunningTime="2026-07-07 17:45:03.679065362 +0000 UTC m=+2255.539145433" Jul 07 17:45:04.636371 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:04.636336 2582 generic.go:356] "Generic (PLEG): container finished" podID="b7d928e9-4877-422a-833b-41685e722ba3" containerID="73fb01ee3598204ac94ff7c93e6e436a7fd39754bf538d11f1fa526131df45e7" exitCode=0 Jul 07 17:45:04.636871 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:04.636423 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" event={"ID":"b7d928e9-4877-422a-833b-41685e722ba3","Type":"ContainerDied","Data":"73fb01ee3598204ac94ff7c93e6e436a7fd39754bf538d11f1fa526131df45e7"} Jul 07 17:45:04.637926 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:04.637901 2582 generic.go:356] "Generic (PLEG): container finished" podID="43b880e6-527e-4d01-8adb-023d41bb7517" containerID="c84f2c99b94904a867c543f1cc86f97889f86fc4f2153418b05bc73ae2f9f81f" exitCode=0 Jul 07 17:45:04.638019 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:04.637981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" event={"ID":"43b880e6-527e-4d01-8adb-023d41bb7517","Type":"ContainerDied","Data":"c84f2c99b94904a867c543f1cc86f97889f86fc4f2153418b05bc73ae2f9f81f"} Jul 07 17:45:04.639216 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:04.639197 2582 generic.go:356] "Generic (PLEG): container finished" podID="28c4e8b0-dc1c-4d2a-b607-a287acab972a" containerID="1012b5a310191114e8ec89a33da9c6b28393b4beec70af4ee489a51095859bea" exitCode=0 Jul 07 17:45:04.639300 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:04.639267 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" event={"ID":"28c4e8b0-dc1c-4d2a-b607-a287acab972a","Type":"ContainerDied","Data":"1012b5a310191114e8ec89a33da9c6b28393b4beec70af4ee489a51095859bea"} Jul 07 17:45:05.077467 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.077374 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:45:05.781290 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.781257 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:05.825266 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.825245 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:05.829324 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.829298 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:05.904841 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.904757 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx7bb\" (UniqueName: \"kubernetes.io/projected/43b880e6-527e-4d01-8adb-023d41bb7517-kube-api-access-gx7bb\") pod \"43b880e6-527e-4d01-8adb-023d41bb7517\" (UID: \"43b880e6-527e-4d01-8adb-023d41bb7517\") " Jul 07 17:45:05.904988 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.904880 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2hbc\" (UniqueName: \"kubernetes.io/projected/b7d928e9-4877-422a-833b-41685e722ba3-kube-api-access-n2hbc\") pod \"b7d928e9-4877-422a-833b-41685e722ba3\" (UID: \"b7d928e9-4877-422a-833b-41685e722ba3\") " Jul 07 17:45:05.904988 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.904904 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2dfj\" (UniqueName: \"kubernetes.io/projected/28c4e8b0-dc1c-4d2a-b607-a287acab972a-kube-api-access-h2dfj\") pod \"28c4e8b0-dc1c-4d2a-b607-a287acab972a\" (UID: \"28c4e8b0-dc1c-4d2a-b607-a287acab972a\") " Jul 07 17:45:05.907009 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.906974 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d928e9-4877-422a-833b-41685e722ba3-kube-api-access-n2hbc" (OuterVolumeSpecName: "kube-api-access-n2hbc") pod "b7d928e9-4877-422a-833b-41685e722ba3" (UID: "b7d928e9-4877-422a-833b-41685e722ba3"). InnerVolumeSpecName "kube-api-access-n2hbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:45:05.907124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.907019 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b880e6-527e-4d01-8adb-023d41bb7517-kube-api-access-gx7bb" (OuterVolumeSpecName: "kube-api-access-gx7bb") pod "43b880e6-527e-4d01-8adb-023d41bb7517" (UID: "43b880e6-527e-4d01-8adb-023d41bb7517"). InnerVolumeSpecName "kube-api-access-gx7bb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:45:05.907124 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:05.907046 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c4e8b0-dc1c-4d2a-b607-a287acab972a-kube-api-access-h2dfj" (OuterVolumeSpecName: "kube-api-access-h2dfj") pod "28c4e8b0-dc1c-4d2a-b607-a287acab972a" (UID: "28c4e8b0-dc1c-4d2a-b607-a287acab972a"). InnerVolumeSpecName "kube-api-access-h2dfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:45:06.005739 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.005703 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hbc\" (UniqueName: \"kubernetes.io/projected/b7d928e9-4877-422a-833b-41685e722ba3-kube-api-access-n2hbc\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:45:06.005739 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.005733 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2dfj\" (UniqueName: \"kubernetes.io/projected/28c4e8b0-dc1c-4d2a-b607-a287acab972a-kube-api-access-h2dfj\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:45:06.005739 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.005742 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx7bb\" (UniqueName: \"kubernetes.io/projected/43b880e6-527e-4d01-8adb-023d41bb7517-kube-api-access-gx7bb\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:45:06.650969 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.650937 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" Jul 07 17:45:06.651157 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.650982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8" event={"ID":"28c4e8b0-dc1c-4d2a-b607-a287acab972a","Type":"ContainerDied","Data":"b9e9bd376f93b168ba1a4cf279e2d9e16976413344609640f391d9019103202e"} Jul 07 17:45:06.651157 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.651028 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9e9bd376f93b168ba1a4cf279e2d9e16976413344609640f391d9019103202e" Jul 07 17:45:06.652307 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.652280 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" Jul 07 17:45:06.652454 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.652284 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-8mmq5" event={"ID":"b7d928e9-4877-422a-833b-41685e722ba3","Type":"ContainerDied","Data":"586554b38f8cc14deaad91d948a8f5b3ad2caa05999a6b2bbe974d3a0dec37f1"} Jul 07 17:45:06.652454 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.652422 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="586554b38f8cc14deaad91d948a8f5b3ad2caa05999a6b2bbe974d3a0dec37f1" Jul 07 17:45:06.653672 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.653646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" event={"ID":"43b880e6-527e-4d01-8adb-023d41bb7517","Type":"ContainerDied","Data":"81ff3575490ed2140defae9efc199bbc8adcaaa45596d83d35e326a622788514"} Jul 07 17:45:06.653775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.653686 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81ff3575490ed2140defae9efc199bbc8adcaaa45596d83d35e326a622788514" Jul 07 17:45:06.653775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:06.653662 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp" Jul 07 17:45:21.166221 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:21.166184 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:45:34.081135 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:34.081092 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:45:54.776730 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:45:54.776690 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:46:14.663876 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:46:14.663838 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:46:34.563906 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:46:34.563812 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:46:47.402865 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:46:47.402823 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:47:04.784068 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:04.784025 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:47:19.080662 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:19.080619 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:47:28.865892 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:28.865860 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:47:28.866689 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:28.866666 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:47:40.482708 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:40.482677 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:47:46.860343 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860307 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7416-139828082150512"] Jul 07 17:47:46.860760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860716 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43b880e6-527e-4d01-8adb-023d41bb7517" containerName="cleanup" Jul 07 17:47:46.860760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860730 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="43b880e6-527e-4d01-8adb-023d41bb7517" containerName="cleanup" Jul 07 17:47:46.860760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860744 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d928e9-4877-422a-833b-41685e722ba3" containerName="cleanup" Jul 07 17:47:46.860760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860749 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d928e9-4877-422a-833b-41685e722ba3" containerName="cleanup" Jul 07 17:47:46.860760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860755 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c4e8b0-dc1c-4d2a-b607-a287acab972a" containerName="cleanup" Jul 07 17:47:46.860760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860761 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c4e8b0-dc1c-4d2a-b607-a287acab972a" containerName="cleanup" Jul 07 17:47:46.860950 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860835 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c4e8b0-dc1c-4d2a-b607-a287acab972a" containerName="cleanup" Jul 07 17:47:46.860950 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860844 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="43b880e6-527e-4d01-8adb-023d41bb7517" containerName="cleanup" Jul 07 17:47:46.860950 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.860849 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d928e9-4877-422a-833b-41685e722ba3" containerName="cleanup" Jul 07 17:47:46.864172 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.864156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:46.867097 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.867075 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wbn2x\"" Jul 07 17:47:46.872176 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.872153 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828082150512"] Jul 07 17:47:46.916488 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:46.916463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-747qk\" (UniqueName: \"kubernetes.io/projected/fa564902-5768-4bc6-b314-d3771c2f05ce-kube-api-access-747qk\") pod \"test-curl-7416-139828082150512\" (UID: \"fa564902-5768-4bc6-b314-d3771c2f05ce\") " pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:47.017388 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:47.017351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-747qk\" (UniqueName: \"kubernetes.io/projected/fa564902-5768-4bc6-b314-d3771c2f05ce-kube-api-access-747qk\") pod \"test-curl-7416-139828082150512\" (UID: \"fa564902-5768-4bc6-b314-d3771c2f05ce\") " pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:47.028015 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:47.027988 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-747qk\" (UniqueName: \"kubernetes.io/projected/fa564902-5768-4bc6-b314-d3771c2f05ce-kube-api-access-747qk\") pod \"test-curl-7416-139828082150512\" (UID: \"fa564902-5768-4bc6-b314-d3771c2f05ce\") " pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:47.189424 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:47.189330 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:47.332678 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:47.332654 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828082150512"] Jul 07 17:47:47.335528 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:47:47.335501 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa564902_5768_4bc6_b314_d3771c2f05ce.slice/crio-f6a322f1f5017372d8cf22e3110143abf261a31a94f5aa534853f3c6e83aae39 WatchSource:0}: Error finding container f6a322f1f5017372d8cf22e3110143abf261a31a94f5aa534853f3c6e83aae39: Status 404 returned error can't find the container with id f6a322f1f5017372d8cf22e3110143abf261a31a94f5aa534853f3c6e83aae39 Jul 07 17:47:47.443684 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:47.443597 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828082150512" event={"ID":"fa564902-5768-4bc6-b314-d3771c2f05ce","Type":"ContainerStarted","Data":"f6a322f1f5017372d8cf22e3110143abf261a31a94f5aa534853f3c6e83aae39"} Jul 07 17:47:48.451000 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:48.450963 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa564902-5768-4bc6-b314-d3771c2f05ce" containerID="b10644e4cd96cb5bfc349d458224fe036452ee95c477c0a9b627bf8e7084a26f" exitCode=0 Jul 07 17:47:48.451408 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:48.451011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828082150512" event={"ID":"fa564902-5768-4bc6-b314-d3771c2f05ce","Type":"ContainerDied","Data":"b10644e4cd96cb5bfc349d458224fe036452ee95c477c0a9b627bf8e7084a26f"} Jul 07 17:47:49.594045 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.594016 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:49.633393 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.633363 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7416-139828082150512_fa564902-5768-4bc6-b314-d3771c2f05ce/test-curl-7416-139828082150512/0.log" Jul 07 17:47:49.643595 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.643559 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-747qk\" (UniqueName: \"kubernetes.io/projected/fa564902-5768-4bc6-b314-d3771c2f05ce-kube-api-access-747qk\") pod \"fa564902-5768-4bc6-b314-d3771c2f05ce\" (UID: \"fa564902-5768-4bc6-b314-d3771c2f05ce\") " Jul 07 17:47:49.645619 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.645595 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa564902-5768-4bc6-b314-d3771c2f05ce-kube-api-access-747qk" (OuterVolumeSpecName: "kube-api-access-747qk") pod "fa564902-5768-4bc6-b314-d3771c2f05ce" (UID: "fa564902-5768-4bc6-b314-d3771c2f05ce"). InnerVolumeSpecName "kube-api-access-747qk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:47:49.666723 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.666700 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7416-139828082150512"] Jul 07 17:47:49.670265 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.670241 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7416-139828082150512"] Jul 07 17:47:49.744880 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.744804 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-747qk\" (UniqueName: \"kubernetes.io/projected/fa564902-5768-4bc6-b314-d3771c2f05ce-kube-api-access-747qk\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:47:49.804595 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.804548 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7416-139828082150624"] Jul 07 17:47:49.804995 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.804981 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa564902-5768-4bc6-b314-d3771c2f05ce" containerName="test-curl-7416-139828082150512" Jul 07 17:47:49.805045 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.804997 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa564902-5768-4bc6-b314-d3771c2f05ce" containerName="test-curl-7416-139828082150512" Jul 07 17:47:49.805083 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.805078 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa564902-5768-4bc6-b314-d3771c2f05ce" containerName="test-curl-7416-139828082150512" Jul 07 17:47:49.808145 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.808127 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:49.814086 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.814063 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828082150624"] Jul 07 17:47:49.946958 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:49.946916 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qltrt\" (UniqueName: \"kubernetes.io/projected/349fb4a2-9e4e-4d2b-8e22-5665256a2810-kube-api-access-qltrt\") pod \"test-curl-7416-139828082150624\" (UID: \"349fb4a2-9e4e-4d2b-8e22-5665256a2810\") " pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:50.048616 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.048496 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qltrt\" (UniqueName: \"kubernetes.io/projected/349fb4a2-9e4e-4d2b-8e22-5665256a2810-kube-api-access-qltrt\") pod \"test-curl-7416-139828082150624\" (UID: \"349fb4a2-9e4e-4d2b-8e22-5665256a2810\") " pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:50.058937 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.058914 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qltrt\" (UniqueName: \"kubernetes.io/projected/349fb4a2-9e4e-4d2b-8e22-5665256a2810-kube-api-access-qltrt\") pod \"test-curl-7416-139828082150624\" (UID: \"349fb4a2-9e4e-4d2b-8e22-5665256a2810\") " pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:50.118787 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.118753 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:50.248798 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.248770 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828082150624"] Jul 07 17:47:50.249970 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:47:50.249938 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod349fb4a2_9e4e_4d2b_8e22_5665256a2810.slice/crio-186de6979f247fb1b914b9050caaf26d2c9fdf56f87627301c2fa103d0b1a3cd WatchSource:0}: Error finding container 186de6979f247fb1b914b9050caaf26d2c9fdf56f87627301c2fa103d0b1a3cd: Status 404 returned error can't find the container with id 186de6979f247fb1b914b9050caaf26d2c9fdf56f87627301c2fa103d0b1a3cd Jul 07 17:47:50.463760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.463718 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828082150624" event={"ID":"349fb4a2-9e4e-4d2b-8e22-5665256a2810","Type":"ContainerStarted","Data":"186de6979f247fb1b914b9050caaf26d2c9fdf56f87627301c2fa103d0b1a3cd"} Jul 07 17:47:50.464943 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.464918 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6a322f1f5017372d8cf22e3110143abf261a31a94f5aa534853f3c6e83aae39" Jul 07 17:47:50.464943 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.464923 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:50.475706 ip-10-0-133-16 kubenswrapper[2582]: E0707 17:47:50.475679 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7416-139828082150512\" is forbidden: User \"system:node:ip-10-0-133-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-16.ec2.internal' and this object" podUID="fa564902-5768-4bc6-b314-d3771c2f05ce" pod="opendatahub/test-curl-7416-139828082150512" Jul 07 17:47:50.935218 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:50.935185 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa564902-5768-4bc6-b314-d3771c2f05ce" path="/var/lib/kubelet/pods/fa564902-5768-4bc6-b314-d3771c2f05ce/volumes" Jul 07 17:47:51.471249 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:51.471217 2582 generic.go:356] "Generic (PLEG): container finished" podID="349fb4a2-9e4e-4d2b-8e22-5665256a2810" containerID="4746790ea1ee7f8955ed805d060af982be4fd11635be64460d33fdaeaa6b72f0" exitCode=0 Jul 07 17:47:51.471405 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:51.471307 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828082150624" event={"ID":"349fb4a2-9e4e-4d2b-8e22-5665256a2810","Type":"ContainerDied","Data":"4746790ea1ee7f8955ed805d060af982be4fd11635be64460d33fdaeaa6b72f0"} Jul 07 17:47:52.614028 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.614006 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:52.637887 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.637859 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7416-139828082150624_349fb4a2-9e4e-4d2b-8e22-5665256a2810/test-curl-7416-139828082150624/0.log" Jul 07 17:47:52.666808 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.666779 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7416-139828082150624"] Jul 07 17:47:52.671099 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.671075 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7416-139828082150624"] Jul 07 17:47:52.673242 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.673220 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qltrt\" (UniqueName: \"kubernetes.io/projected/349fb4a2-9e4e-4d2b-8e22-5665256a2810-kube-api-access-qltrt\") pod \"349fb4a2-9e4e-4d2b-8e22-5665256a2810\" (UID: \"349fb4a2-9e4e-4d2b-8e22-5665256a2810\") " Jul 07 17:47:52.675280 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.675259 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349fb4a2-9e4e-4d2b-8e22-5665256a2810-kube-api-access-qltrt" (OuterVolumeSpecName: "kube-api-access-qltrt") pod "349fb4a2-9e4e-4d2b-8e22-5665256a2810" (UID: "349fb4a2-9e4e-4d2b-8e22-5665256a2810"). InnerVolumeSpecName "kube-api-access-qltrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:47:52.774169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.774089 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qltrt\" (UniqueName: \"kubernetes.io/projected/349fb4a2-9e4e-4d2b-8e22-5665256a2810-kube-api-access-qltrt\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:47:52.840804 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.840767 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:52.841179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.841166 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349fb4a2-9e4e-4d2b-8e22-5665256a2810" containerName="test-curl-7416-139828082150624" Jul 07 17:47:52.841233 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.841180 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="349fb4a2-9e4e-4d2b-8e22-5665256a2810" containerName="test-curl-7416-139828082150624" Jul 07 17:47:52.841272 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.841247 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="349fb4a2-9e4e-4d2b-8e22-5665256a2810" containerName="test-curl-7416-139828082150624" Jul 07 17:47:52.844318 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.844301 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:52.853923 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.853898 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:52.934922 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.934887 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="349fb4a2-9e4e-4d2b-8e22-5665256a2810" path="/var/lib/kubelet/pods/349fb4a2-9e4e-4d2b-8e22-5665256a2810/volumes" Jul 07 17:47:52.975447 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:52.975417 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/3abfd310-714d-40d0-adff-7d570b6b025e-kube-api-access-cgnzd\") pod \"test-curl-7416-139828084968016\" (UID: \"3abfd310-714d-40d0-adff-7d570b6b025e\") " pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:53.076430 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.076346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/3abfd310-714d-40d0-adff-7d570b6b025e-kube-api-access-cgnzd\") pod \"test-curl-7416-139828084968016\" (UID: \"3abfd310-714d-40d0-adff-7d570b6b025e\") " pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:53.086546 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.086521 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/3abfd310-714d-40d0-adff-7d570b6b025e-kube-api-access-cgnzd\") pod \"test-curl-7416-139828084968016\" (UID: \"3abfd310-714d-40d0-adff-7d570b6b025e\") " pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:53.154179 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.154145 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:53.287885 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.287862 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:53.290050 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:47:53.290021 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3abfd310_714d_40d0_adff_7d570b6b025e.slice/crio-ce4c9d0c86710d42202daf1098110859c00c2226fd9a8a10175f48d666d912aa WatchSource:0}: Error finding container ce4c9d0c86710d42202daf1098110859c00c2226fd9a8a10175f48d666d912aa: Status 404 returned error can't find the container with id ce4c9d0c86710d42202daf1098110859c00c2226fd9a8a10175f48d666d912aa Jul 07 17:47:53.482666 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.482624 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968016" event={"ID":"3abfd310-714d-40d0-adff-7d570b6b025e","Type":"ContainerStarted","Data":"ce4c9d0c86710d42202daf1098110859c00c2226fd9a8a10175f48d666d912aa"} Jul 07 17:47:53.483879 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.483860 2582 scope.go:117] "RemoveContainer" containerID="4746790ea1ee7f8955ed805d060af982be4fd11635be64460d33fdaeaa6b72f0" Jul 07 17:47:53.483981 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:53.483867 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828082150624" Jul 07 17:47:54.490725 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:54.490688 2582 generic.go:356] "Generic (PLEG): container finished" podID="3abfd310-714d-40d0-adff-7d570b6b025e" containerID="3b0eace2c7a13d12b14092f71f10af2f32e104848e48d42ff6938cb282d5f11b" exitCode=0 Jul 07 17:47:54.491169 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:54.490777 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968016" event={"ID":"3abfd310-714d-40d0-adff-7d570b6b025e","Type":"ContainerDied","Data":"3b0eace2c7a13d12b14092f71f10af2f32e104848e48d42ff6938cb282d5f11b"} Jul 07 17:47:55.635454 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.635426 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:55.656760 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.656734 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7416-139828084968016_3abfd310-714d-40d0-adff-7d570b6b025e/test-curl-7416-139828084968016/0.log" Jul 07 17:47:55.682603 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.682564 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:55.687498 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.687475 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:55.701074 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.701053 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/3abfd310-714d-40d0-adff-7d570b6b025e-kube-api-access-cgnzd\") pod \"3abfd310-714d-40d0-adff-7d570b6b025e\" (UID: \"3abfd310-714d-40d0-adff-7d570b6b025e\") " Jul 07 17:47:55.703100 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.703071 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abfd310-714d-40d0-adff-7d570b6b025e-kube-api-access-cgnzd" (OuterVolumeSpecName: "kube-api-access-cgnzd") pod "3abfd310-714d-40d0-adff-7d570b6b025e" (UID: "3abfd310-714d-40d0-adff-7d570b6b025e"). InnerVolumeSpecName "kube-api-access-cgnzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:47:55.801968 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.801873 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/3abfd310-714d-40d0-adff-7d570b6b025e-kube-api-access-cgnzd\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:47:55.818067 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.818036 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:55.818443 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.818430 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3abfd310-714d-40d0-adff-7d570b6b025e" containerName="test-curl-7416-139828084968016" Jul 07 17:47:55.818486 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.818445 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3abfd310-714d-40d0-adff-7d570b6b025e" containerName="test-curl-7416-139828084968016" Jul 07 17:47:55.818522 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.818507 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3abfd310-714d-40d0-adff-7d570b6b025e" containerName="test-curl-7416-139828084968016" Jul 07 17:47:55.821560 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.821543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:55.839224 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.839198 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:55.902926 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:55.902890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gckdv\" (UniqueName: \"kubernetes.io/projected/35bf7974-1836-4271-bd01-c108005ab39d-kube-api-access-gckdv\") pod \"test-curl-7416-139828084968016\" (UID: \"35bf7974-1836-4271-bd01-c108005ab39d\") " pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:56.003848 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.003815 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gckdv\" (UniqueName: \"kubernetes.io/projected/35bf7974-1836-4271-bd01-c108005ab39d-kube-api-access-gckdv\") pod \"test-curl-7416-139828084968016\" (UID: \"35bf7974-1836-4271-bd01-c108005ab39d\") " pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:56.013706 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.013678 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gckdv\" (UniqueName: \"kubernetes.io/projected/35bf7974-1836-4271-bd01-c108005ab39d-kube-api-access-gckdv\") pod \"test-curl-7416-139828084968016\" (UID: \"35bf7974-1836-4271-bd01-c108005ab39d\") " pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:56.131223 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.131188 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:56.269380 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.269225 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:56.271452 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:47:56.271426 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35bf7974_1836_4271_bd01_c108005ab39d.slice/crio-dca3eb729511ae35e26306b3af903e2cc37a2b93b563fdbd7bf57944d01e4848 WatchSource:0}: Error finding container dca3eb729511ae35e26306b3af903e2cc37a2b93b563fdbd7bf57944d01e4848: Status 404 returned error can't find the container with id dca3eb729511ae35e26306b3af903e2cc37a2b93b563fdbd7bf57944d01e4848 Jul 07 17:47:56.502673 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.502566 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:56.502673 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.502600 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce4c9d0c86710d42202daf1098110859c00c2226fd9a8a10175f48d666d912aa" Jul 07 17:47:56.503888 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.503859 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968016" event={"ID":"35bf7974-1836-4271-bd01-c108005ab39d","Type":"ContainerStarted","Data":"dca3eb729511ae35e26306b3af903e2cc37a2b93b563fdbd7bf57944d01e4848"} Jul 07 17:47:56.515822 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.515794 2582 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7416-139828084968016" oldPodUID="3abfd310-714d-40d0-adff-7d570b6b025e" podUID="35bf7974-1836-4271-bd01-c108005ab39d" Jul 07 17:47:56.777003 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.776919 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:47:56.935084 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:56.935050 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3abfd310-714d-40d0-adff-7d570b6b025e" path="/var/lib/kubelet/pods/3abfd310-714d-40d0-adff-7d570b6b025e/volumes" Jul 07 17:47:57.510482 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:57.510452 2582 generic.go:356] "Generic (PLEG): container finished" podID="35bf7974-1836-4271-bd01-c108005ab39d" containerID="5c4363d3dea911ca5f1bb0800f16b65cc700f140516b24a4b4775542abab62e6" exitCode=0 Jul 07 17:47:57.510684 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:57.510547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968016" event={"ID":"35bf7974-1836-4271-bd01-c108005ab39d","Type":"ContainerDied","Data":"5c4363d3dea911ca5f1bb0800f16b65cc700f140516b24a4b4775542abab62e6"} Jul 07 17:47:58.655455 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.655430 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:47:58.678516 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.678489 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7416-139828084968016_35bf7974-1836-4271-bd01-c108005ab39d/test-curl-7416-139828084968016/0.log" Jul 07 17:47:58.707712 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.707680 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:58.712497 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.712476 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7416-139828084968016"] Jul 07 17:47:58.726355 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.726336 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gckdv\" (UniqueName: \"kubernetes.io/projected/35bf7974-1836-4271-bd01-c108005ab39d-kube-api-access-gckdv\") pod \"35bf7974-1836-4271-bd01-c108005ab39d\" (UID: \"35bf7974-1836-4271-bd01-c108005ab39d\") " Jul 07 17:47:58.728345 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.728312 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35bf7974-1836-4271-bd01-c108005ab39d-kube-api-access-gckdv" (OuterVolumeSpecName: "kube-api-access-gckdv") pod "35bf7974-1836-4271-bd01-c108005ab39d" (UID: "35bf7974-1836-4271-bd01-c108005ab39d"). InnerVolumeSpecName "kube-api-access-gckdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:47:58.828033 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.827955 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gckdv\" (UniqueName: \"kubernetes.io/projected/35bf7974-1836-4271-bd01-c108005ab39d-kube-api-access-gckdv\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:47:58.845088 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.845063 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:47:58.845499 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.845485 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35bf7974-1836-4271-bd01-c108005ab39d" containerName="test-curl-7416-139828084968016" Jul 07 17:47:58.845550 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.845502 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="35bf7974-1836-4271-bd01-c108005ab39d" containerName="test-curl-7416-139828084968016" Jul 07 17:47:58.845611 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.845564 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="35bf7974-1836-4271-bd01-c108005ab39d" containerName="test-curl-7416-139828084968016" Jul 07 17:47:58.848618 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.848603 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:47:58.855527 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.855291 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:47:58.928541 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.928501 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62mzh\" (UniqueName: \"kubernetes.io/projected/d5daad19-3b93-4cbe-b938-0da6a1fc64e6-kube-api-access-62mzh\") pod \"test-curl-7416-139828084968736\" (UID: \"d5daad19-3b93-4cbe-b938-0da6a1fc64e6\") " pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:47:58.934427 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:58.934395 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35bf7974-1836-4271-bd01-c108005ab39d" path="/var/lib/kubelet/pods/35bf7974-1836-4271-bd01-c108005ab39d/volumes" Jul 07 17:47:59.029193 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.029156 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62mzh\" (UniqueName: \"kubernetes.io/projected/d5daad19-3b93-4cbe-b938-0da6a1fc64e6-kube-api-access-62mzh\") pod \"test-curl-7416-139828084968736\" (UID: \"d5daad19-3b93-4cbe-b938-0da6a1fc64e6\") " pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:47:59.042036 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.042010 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62mzh\" (UniqueName: \"kubernetes.io/projected/d5daad19-3b93-4cbe-b938-0da6a1fc64e6-kube-api-access-62mzh\") pod \"test-curl-7416-139828084968736\" (UID: \"d5daad19-3b93-4cbe-b938-0da6a1fc64e6\") " pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:47:59.158954 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.158919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:47:59.304560 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.304536 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:47:59.307168 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:47:59.307137 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5daad19_3b93_4cbe_b938_0da6a1fc64e6.slice/crio-e7827f9faf81e6f683e3a64b49e43a0e330e1f248f566ce8ea922c1dc4ec5a4f WatchSource:0}: Error finding container e7827f9faf81e6f683e3a64b49e43a0e330e1f248f566ce8ea922c1dc4ec5a4f: Status 404 returned error can't find the container with id e7827f9faf81e6f683e3a64b49e43a0e330e1f248f566ce8ea922c1dc4ec5a4f Jul 07 17:47:59.521962 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.521878 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968736" event={"ID":"d5daad19-3b93-4cbe-b938-0da6a1fc64e6","Type":"ContainerStarted","Data":"e7827f9faf81e6f683e3a64b49e43a0e330e1f248f566ce8ea922c1dc4ec5a4f"} Jul 07 17:47:59.523201 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.523182 2582 scope.go:117] "RemoveContainer" containerID="5c4363d3dea911ca5f1bb0800f16b65cc700f140516b24a4b4775542abab62e6" Jul 07 17:47:59.523310 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:47:59.523182 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968016" Jul 07 17:48:00.530853 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:00.530817 2582 generic.go:356] "Generic (PLEG): container finished" podID="d5daad19-3b93-4cbe-b938-0da6a1fc64e6" containerID="206cc3e87a787740c045e87f5c566c25383a4ab1a7d33b3b921807ad94d02525" exitCode=0 Jul 07 17:48:00.531245 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:00.530912 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968736" event={"ID":"d5daad19-3b93-4cbe-b938-0da6a1fc64e6","Type":"ContainerDied","Data":"206cc3e87a787740c045e87f5c566c25383a4ab1a7d33b3b921807ad94d02525"} Jul 07 17:48:01.678477 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.678451 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:01.706931 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.706901 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7416-139828084968736_d5daad19-3b93-4cbe-b938-0da6a1fc64e6/test-curl-7416-139828084968736/0.log" Jul 07 17:48:01.735467 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.735435 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:01.741025 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.740999 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:01.756340 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.756314 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62mzh\" (UniqueName: \"kubernetes.io/projected/d5daad19-3b93-4cbe-b938-0da6a1fc64e6-kube-api-access-62mzh\") pod \"d5daad19-3b93-4cbe-b938-0da6a1fc64e6\" (UID: \"d5daad19-3b93-4cbe-b938-0da6a1fc64e6\") " Jul 07 17:48:01.758391 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.758362 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5daad19-3b93-4cbe-b938-0da6a1fc64e6-kube-api-access-62mzh" (OuterVolumeSpecName: "kube-api-access-62mzh") pod "d5daad19-3b93-4cbe-b938-0da6a1fc64e6" (UID: "d5daad19-3b93-4cbe-b938-0da6a1fc64e6"). InnerVolumeSpecName "kube-api-access-62mzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:01.857410 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.857323 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62mzh\" (UniqueName: \"kubernetes.io/projected/d5daad19-3b93-4cbe-b938-0da6a1fc64e6-kube-api-access-62mzh\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:01.864802 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.864773 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:01.865172 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.865159 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5daad19-3b93-4cbe-b938-0da6a1fc64e6" containerName="test-curl-7416-139828084968736" Jul 07 17:48:01.865267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.865173 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5daad19-3b93-4cbe-b938-0da6a1fc64e6" containerName="test-curl-7416-139828084968736" Jul 07 17:48:01.865267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.865256 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5daad19-3b93-4cbe-b938-0da6a1fc64e6" containerName="test-curl-7416-139828084968736" Jul 07 17:48:01.868250 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.868234 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:01.876912 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.876884 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:01.958745 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:01.958708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkpc4\" (UniqueName: \"kubernetes.io/projected/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9-kube-api-access-mkpc4\") pod \"test-curl-7416-139828084968736\" (UID: \"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9\") " pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:02.059925 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.059879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkpc4\" (UniqueName: \"kubernetes.io/projected/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9-kube-api-access-mkpc4\") pod \"test-curl-7416-139828084968736\" (UID: \"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9\") " pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:02.070138 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.070113 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkpc4\" (UniqueName: \"kubernetes.io/projected/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9-kube-api-access-mkpc4\") pod \"test-curl-7416-139828084968736\" (UID: \"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9\") " pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:02.178359 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.178322 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:02.336185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.336156 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:02.338194 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:48:02.338167 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7562bba5_ab97_4cc4_aa73_c0b819a6f9e9.slice/crio-67dcd361cce1db5561ef2b0e33e87bf02f73335867f7196d3aba4c84d48c9fd2 WatchSource:0}: Error finding container 67dcd361cce1db5561ef2b0e33e87bf02f73335867f7196d3aba4c84d48c9fd2: Status 404 returned error can't find the container with id 67dcd361cce1db5561ef2b0e33e87bf02f73335867f7196d3aba4c84d48c9fd2 Jul 07 17:48:02.543658 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.543545 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:02.543658 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.543607 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7827f9faf81e6f683e3a64b49e43a0e330e1f248f566ce8ea922c1dc4ec5a4f" Jul 07 17:48:02.551861 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.551822 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968736" event={"ID":"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9","Type":"ContainerStarted","Data":"67dcd361cce1db5561ef2b0e33e87bf02f73335867f7196d3aba4c84d48c9fd2"} Jul 07 17:48:02.560164 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.560138 2582 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7416-139828084968736" oldPodUID="d5daad19-3b93-4cbe-b938-0da6a1fc64e6" podUID="7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" Jul 07 17:48:02.934246 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:02.934212 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5daad19-3b93-4cbe-b938-0da6a1fc64e6" path="/var/lib/kubelet/pods/d5daad19-3b93-4cbe-b938-0da6a1fc64e6/volumes" Jul 07 17:48:03.558341 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:03.558302 2582 generic.go:356] "Generic (PLEG): container finished" podID="7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" containerID="0a8f5ce64206b63ea282d787a4f4367bee341f476dd8364a7c455fc2a99cc69e" exitCode=0 Jul 07 17:48:03.558515 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:03.558393 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7416-139828084968736" event={"ID":"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9","Type":"ContainerDied","Data":"0a8f5ce64206b63ea282d787a4f4367bee341f476dd8364a7c455fc2a99cc69e"} Jul 07 17:48:04.705223 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.705200 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:04.731696 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.731646 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7416-139828084968736_7562bba5-ab97-4cc4-aa73-c0b819a6f9e9/test-curl-7416-139828084968736/0.log" Jul 07 17:48:04.761925 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.761890 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:04.766654 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.766629 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7416-139828084968736"] Jul 07 17:48:04.787058 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.787033 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkpc4\" (UniqueName: \"kubernetes.io/projected/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9-kube-api-access-mkpc4\") pod \"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9\" (UID: \"7562bba5-ab97-4cc4-aa73-c0b819a6f9e9\") " Jul 07 17:48:04.789110 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.789083 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9-kube-api-access-mkpc4" (OuterVolumeSpecName: "kube-api-access-mkpc4") pod "7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" (UID: "7562bba5-ab97-4cc4-aa73-c0b819a6f9e9"). InnerVolumeSpecName "kube-api-access-mkpc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:04.888641 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.888605 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkpc4\" (UniqueName: \"kubernetes.io/projected/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9-kube-api-access-mkpc4\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:04.934441 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:04.934410 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" path="/var/lib/kubelet/pods/7562bba5-ab97-4cc4-aa73-c0b819a6f9e9/volumes" Jul 07 17:48:05.569911 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:05.569880 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7416-139828084968736" Jul 07 17:48:05.569911 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:05.569898 2582 scope.go:117] "RemoveContainer" containerID="0a8f5ce64206b63ea282d787a4f4367bee341f476dd8364a7c455fc2a99cc69e" Jul 07 17:48:06.091711 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:06.091680 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8"] Jul 07 17:48:06.101267 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:06.101243 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-19c9b1-29724105-mqfg8"] Jul 07 17:48:06.934463 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:06.934429 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c4e8b0-dc1c-4d2a-b607-a287acab972a" path="/var/lib/kubelet/pods/28c4e8b0-dc1c-4d2a-b607-a287acab972a/volumes" Jul 07 17:48:11.193140 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:11.193103 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:48:14.363178 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:14.363142 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp"] Jul 07 17:48:14.368707 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:14.368684 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-ce87c9-29724105-k65zp"] Jul 07 17:48:14.934093 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:14.934056 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43b880e6-527e-4d01-8adb-023d41bb7517" path="/var/lib/kubelet/pods/43b880e6-527e-4d01-8adb-023d41bb7517/volumes" Jul 07 17:48:21.525152 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:21.525121 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5"] Jul 07 17:48:21.525758 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:21.525356 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" containerID="cri-o://93f06e409c4f06a5c02adfa91f2b15bb93f7fe9ce9e57e1e3dd14d56cb5cc1af" gracePeriod=30 Jul 07 17:48:23.582006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:23.581972 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xpt95"] Jul 07 17:48:26.683772 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.683738 2582 generic.go:356] "Generic (PLEG): container finished" podID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerID="93f06e409c4f06a5c02adfa91f2b15bb93f7fe9ce9e57e1e3dd14d56cb5cc1af" exitCode=0 Jul 07 17:48:26.684258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.683789 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" event={"ID":"d3230a74-3332-4fd3-a8db-68e2bd1865ff","Type":"ContainerDied","Data":"93f06e409c4f06a5c02adfa91f2b15bb93f7fe9ce9e57e1e3dd14d56cb5cc1af"} Jul 07 17:48:26.684258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.683851 2582 scope.go:117] "RemoveContainer" containerID="80cb44d66bc2a3d26d7f598e97d6057b5b6a6d2abfa065d70065f4740cf1b582" Jul 07 17:48:26.794488 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.794428 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:48:26.901486 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901452 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-socket\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901690 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901495 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p59fj\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-kube-api-access-p59fj\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901690 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901517 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-credential-socket\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901690 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901545 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istiod-ca-cert\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901740 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-data\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901788 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-token\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901822 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-podinfo\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.901866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901845 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:26.901866 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901858 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:26.902114 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.901862 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-envoy\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.902114 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902005 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-certs\") pod \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\" (UID: \"d3230a74-3332-4fd3-a8db-68e2bd1865ff\") " Jul 07 17:48:26.902114 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902010 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:48:26.902288 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902242 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:26.902445 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902410 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-certs\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:26.902445 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902432 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-workload-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:26.902445 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902448 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-credential-socket\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:26.902700 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902461 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istiod-ca-cert\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:26.902700 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.902511 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:26.904006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.903986 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 17:48:26.904006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.903998 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:26.904281 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.904261 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:26.904370 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:26.904285 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-kube-api-access-p59fj" (OuterVolumeSpecName: "kube-api-access-p59fj") pod "d3230a74-3332-4fd3-a8db-68e2bd1865ff" (UID: "d3230a74-3332-4fd3-a8db-68e2bd1865ff"). InnerVolumeSpecName "kube-api-access-p59fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:27.003185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.003146 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-podinfo\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.003185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.003176 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-envoy\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.003185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.003185 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p59fj\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-kube-api-access-p59fj\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.003185 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.003195 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-data\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.003451 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.003204 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3230a74-3332-4fd3-a8db-68e2bd1865ff-istio-token\") on node \"ip-10-0-133-16.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.219693 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.219663 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z5btm_0b9b46ee-c52a-4b30-a95b-96c85e84c51b/manager/0.log" Jul 07 17:48:27.361909 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.361879 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zcmnd_61271ea3-07a6-4510-bca4-5bfdf8616073/manager/0.log" Jul 07 17:48:27.695914 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.695883 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" Jul 07 17:48:27.696323 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.695882 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5" event={"ID":"d3230a74-3332-4fd3-a8db-68e2bd1865ff","Type":"ContainerDied","Data":"a22227312fa65d79930fb58f7697b0fdcf8ddb36212ac71e21e7f6c6c30fa39e"} Jul 07 17:48:27.696323 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.696043 2582 scope.go:117] "RemoveContainer" containerID="93f06e409c4f06a5c02adfa91f2b15bb93f7fe9ce9e57e1e3dd14d56cb5cc1af" Jul 07 17:48:27.721264 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.721241 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5"] Jul 07 17:48:27.729833 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.729812 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ce87c9-openshift-default-bd5c66779-wmnw5"] Jul 07 17:48:27.742839 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:27.742817 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724105-8mmq5_b7d928e9-4877-422a-833b-41685e722ba3/cleanup/0.log" Jul 07 17:48:28.401941 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:28.401910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-ktth7_8119380d-de5a-46dd-a204-ca61cc80e57b/copy-manifests/0.log" Jul 07 17:48:28.410977 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:28.410943 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-ktth7_8119380d-de5a-46dd-a204-ca61cc80e57b/manager/0.log" Jul 07 17:48:28.934733 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:28.934700 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" path="/var/lib/kubelet/pods/d3230a74-3332-4fd3-a8db-68e2bd1865ff/volumes" Jul 07 17:48:30.887118 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:30.887086 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xpt95_6460b40e-7a6f-4e03-952b-d81aaabcacb0/limitador/0.log" Jul 07 17:48:31.403837 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:31.403806 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45rdln_28d6ce2b-c497-4b33-b5a2-2447b66361f5/istio-proxy/0.log" Jul 07 17:48:31.655066 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:31.654985 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-4wv8v_3cb9abe8-8b1e-4b48-93ae-b043104a9c43/kube-auth-proxy/0.log" Jul 07 17:48:38.207292 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207261 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47h5d/must-gather-9mb75"] Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207656 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207667 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207679 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" containerName="test-curl-7416-139828084968736" Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207685 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" containerName="test-curl-7416-139828084968736" Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207755 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207764 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" Jul 07 17:48:38.207768 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207771 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7562bba5-ab97-4cc4-aa73-c0b819a6f9e9" containerName="test-curl-7416-139828084968736" Jul 07 17:48:38.207990 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207822 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" Jul 07 17:48:38.207990 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.207829 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3230a74-3332-4fd3-a8db-68e2bd1865ff" containerName="istio-proxy" Jul 07 17:48:38.210657 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.210641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.212942 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.212918 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47h5d\"/\"openshift-service-ca.crt\"" Jul 07 17:48:38.213076 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.212994 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-47h5d\"/\"default-dockercfg-mt88j\"" Jul 07 17:48:38.213775 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.213756 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47h5d\"/\"kube-root-ca.crt\"" Jul 07 17:48:38.225059 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.225035 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47h5d/must-gather-9mb75"] Jul 07 17:48:38.307548 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.307511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z59h\" (UniqueName: \"kubernetes.io/projected/6ecd511e-721c-438f-a2a4-35b27f6c7aea-kube-api-access-4z59h\") pod \"must-gather-9mb75\" (UID: \"6ecd511e-721c-438f-a2a4-35b27f6c7aea\") " pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.307734 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.307583 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6ecd511e-721c-438f-a2a4-35b27f6c7aea-must-gather-output\") pod \"must-gather-9mb75\" (UID: \"6ecd511e-721c-438f-a2a4-35b27f6c7aea\") " pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.408307 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.408272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z59h\" (UniqueName: \"kubernetes.io/projected/6ecd511e-721c-438f-a2a4-35b27f6c7aea-kube-api-access-4z59h\") pod \"must-gather-9mb75\" (UID: \"6ecd511e-721c-438f-a2a4-35b27f6c7aea\") " pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.408493 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.408350 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6ecd511e-721c-438f-a2a4-35b27f6c7aea-must-gather-output\") pod \"must-gather-9mb75\" (UID: \"6ecd511e-721c-438f-a2a4-35b27f6c7aea\") " pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.408711 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.408692 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6ecd511e-721c-438f-a2a4-35b27f6c7aea-must-gather-output\") pod \"must-gather-9mb75\" (UID: \"6ecd511e-721c-438f-a2a4-35b27f6c7aea\") " pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.418536 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.418505 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z59h\" (UniqueName: \"kubernetes.io/projected/6ecd511e-721c-438f-a2a4-35b27f6c7aea-kube-api-access-4z59h\") pod \"must-gather-9mb75\" (UID: \"6ecd511e-721c-438f-a2a4-35b27f6c7aea\") " pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.536546 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.536460 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47h5d/must-gather-9mb75" Jul 07 17:48:38.671231 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.671202 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47h5d/must-gather-9mb75"] Jul 07 17:48:38.672864 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:48:38.672836 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ecd511e_721c_438f_a2a4_35b27f6c7aea.slice/crio-91bb8e1459ce20480639c2686565deb381edaac90753a515b695c8e13ce9fd2f WatchSource:0}: Error finding container 91bb8e1459ce20480639c2686565deb381edaac90753a515b695c8e13ce9fd2f: Status 404 returned error can't find the container with id 91bb8e1459ce20480639c2686565deb381edaac90753a515b695c8e13ce9fd2f Jul 07 17:48:38.757037 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:38.757000 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/must-gather-9mb75" event={"ID":"6ecd511e-721c-438f-a2a4-35b27f6c7aea","Type":"ContainerStarted","Data":"91bb8e1459ce20480639c2686565deb381edaac90753a515b695c8e13ce9fd2f"} Jul 07 17:48:39.766191 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:39.766061 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/must-gather-9mb75" event={"ID":"6ecd511e-721c-438f-a2a4-35b27f6c7aea","Type":"ContainerStarted","Data":"51c6b35363b10cd5a699a2eda574bf164cf662eedd7093342f95ee273fca9357"} Jul 07 17:48:39.766191 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:39.766144 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/must-gather-9mb75" event={"ID":"6ecd511e-721c-438f-a2a4-35b27f6c7aea","Type":"ContainerStarted","Data":"cff63fd2852c9822ae358f2505def33a5669a28d2c47150db9933a22316c009e"} Jul 07 17:48:39.786859 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:39.786789 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-47h5d/must-gather-9mb75" podStartSLOduration=1.07548887 podStartE2EDuration="1.786767112s" totalImagesPullingTime="711.278242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:48:39.784765863 +0000 UTC m=+2471.644845945" watchObservedRunningTime="2026-07-07 17:48:39.786767112 +0000 UTC m=+2471.646847186" Jul 07 17:48:41.100946 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:41.100906 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-442hr_039a7f67-f570-4358-8404-083823f9c823/global-pull-secret-syncer/0.log" Jul 07 17:48:41.308006 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:41.307970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5sj99_e1387f8c-3973-4bbf-8f83-a0d502302121/konnectivity-agent/0.log" Jul 07 17:48:41.368060 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:41.368024 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-16.ec2.internal_b956c94c524f1281d984172ee6c41e32/haproxy/0.log" Jul 07 17:48:46.472276 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:46.472219 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xpt95_6460b40e-7a6f-4e03-952b-d81aaabcacb0/limitador/0.log" Jul 07 17:48:48.645924 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.645880 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-jxddj_c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a/kube-state-metrics/0.log" Jul 07 17:48:48.669089 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.669051 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-jxddj_c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a/kube-rbac-proxy-main/0.log" Jul 07 17:48:48.707041 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.706997 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-jxddj_c73e2d5b-9f13-4165-bb4b-c4a6f2d1062a/kube-rbac-proxy-self/0.log" Jul 07 17:48:48.743649 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.743618 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-66b546c5b5-nsc5n_9c80680a-17b4-40fa-b64c-b324dcb57341/metrics-server/0.log" Jul 07 17:48:48.823633 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.823600 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gcznj_8a1d7fd6-2578-446b-a21f-61ed365b16c7/node-exporter/0.log" Jul 07 17:48:48.854278 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.854246 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gcznj_8a1d7fd6-2578-446b-a21f-61ed365b16c7/kube-rbac-proxy/0.log" Jul 07 17:48:48.885491 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:48.885461 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gcznj_8a1d7fd6-2578-446b-a21f-61ed365b16c7/init-textfile/0.log" Jul 07 17:48:49.118730 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.118703 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8fxtk_8e7a1707-3649-415f-9049-742731e1172a/kube-rbac-proxy-main/0.log" Jul 07 17:48:49.151335 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.151296 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8fxtk_8e7a1707-3649-415f-9049-742731e1172a/kube-rbac-proxy-self/0.log" Jul 07 17:48:49.187063 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.187034 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8fxtk_8e7a1707-3649-415f-9049-742731e1172a/openshift-state-metrics/0.log" Jul 07 17:48:49.243647 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.243617 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/prometheus/0.log" Jul 07 17:48:49.277307 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.277267 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/config-reloader/0.log" Jul 07 17:48:49.311674 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.311642 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/thanos-sidecar/0.log" Jul 07 17:48:49.343953 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.343904 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/kube-rbac-proxy-web/0.log" Jul 07 17:48:49.371100 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.371013 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/kube-rbac-proxy/0.log" Jul 07 17:48:49.399955 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.399906 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/kube-rbac-proxy-thanos/0.log" Jul 07 17:48:49.429157 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.429121 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd28042-1af8-4973-aea3-d27764a33de4/init-config-reloader/0.log" Jul 07 17:48:49.529828 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.529796 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-xndkl_ac5ac622-54ce-40b0-9d0b-10dcbf7f6631/prometheus-operator-admission-webhook/0.log" Jul 07 17:48:49.564338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.564300 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-688bfdc6b7-726h2_4e23bade-2f11-45f0-993f-8fc9e903fd7d/telemeter-client/0.log" Jul 07 17:48:49.596308 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.596278 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-688bfdc6b7-726h2_4e23bade-2f11-45f0-993f-8fc9e903fd7d/reload/0.log" Jul 07 17:48:49.622210 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.622128 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-688bfdc6b7-726h2_4e23bade-2f11-45f0-993f-8fc9e903fd7d/kube-rbac-proxy/0.log" Jul 07 17:48:49.657061 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.657022 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b79df7448-nbddj_ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1/thanos-query/0.log" Jul 07 17:48:49.683610 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.683561 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b79df7448-nbddj_ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1/kube-rbac-proxy-web/0.log" Jul 07 17:48:49.706922 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.706891 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b79df7448-nbddj_ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1/kube-rbac-proxy/0.log" Jul 07 17:48:49.730752 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.730712 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b79df7448-nbddj_ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1/prom-label-proxy/0.log" Jul 07 17:48:49.755232 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.755198 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b79df7448-nbddj_ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1/kube-rbac-proxy-rules/0.log" Jul 07 17:48:49.780671 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:49.780641 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b79df7448-nbddj_ca6a3c0c-7b17-4f8a-bf9c-7fed19944ee1/kube-rbac-proxy-metrics/0.log" Jul 07 17:48:51.019542 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.019501 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7"] Jul 07 17:48:51.034887 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.032637 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-446rp_70f4fece-813b-4fff-aa86-06d7c1ca0575/networking-console-plugin/0.log" Jul 07 17:48:51.036324 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.036279 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.036732 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.036650 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7"] Jul 07 17:48:51.047477 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.047082 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-sys\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.047477 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.047202 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hz4s\" (UniqueName: \"kubernetes.io/projected/29662683-b575-4e21-a54d-6cf839531ffd-kube-api-access-9hz4s\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.047477 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.047247 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-lib-modules\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.047477 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.047307 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-proc\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.047477 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.047373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-podres\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148153 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-sys\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148195 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hz4s\" (UniqueName: \"kubernetes.io/projected/29662683-b575-4e21-a54d-6cf839531ffd-kube-api-access-9hz4s\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148226 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-lib-modules\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148229 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-sys\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148277 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-proc\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148338 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148310 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-podres\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148678 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148378 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-lib-modules\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148678 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148456 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-podres\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.148678 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.148509 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/29662683-b575-4e21-a54d-6cf839531ffd-proc\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.159379 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.159351 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hz4s\" (UniqueName: \"kubernetes.io/projected/29662683-b575-4e21-a54d-6cf839531ffd-kube-api-access-9hz4s\") pod \"perf-node-gather-daemonset-4rdr7\" (UID: \"29662683-b575-4e21-a54d-6cf839531ffd\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.359303 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.359221 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.542960 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.542738 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7"] Jul 07 17:48:51.548592 ip-10-0-133-16 kubenswrapper[2582]: W0707 17:48:51.547496 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod29662683_b575_4e21_a54d_6cf839531ffd.slice/crio-bc150e1698befe468fd5b280f183b0ac8b217aca050208baa2c2c9b02936a380 WatchSource:0}: Error finding container bc150e1698befe468fd5b280f183b0ac8b217aca050208baa2c2c9b02936a380: Status 404 returned error can't find the container with id bc150e1698befe468fd5b280f183b0ac8b217aca050208baa2c2c9b02936a380 Jul 07 17:48:51.756903 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.756836 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/2.log" Jul 07 17:48:51.764544 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.764510 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-frwl4_346488bd-0761-4f1f-a947-c476615e2deb/console-operator/3.log" Jul 07 17:48:51.855965 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.855932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" event={"ID":"29662683-b575-4e21-a54d-6cf839531ffd","Type":"ContainerStarted","Data":"7d9177e758492750113e2371e3f03e5cd0452c6f19d845f500a0dbc5e6c4589e"} Jul 07 17:48:51.856236 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.855991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" event={"ID":"29662683-b575-4e21-a54d-6cf839531ffd","Type":"ContainerStarted","Data":"bc150e1698befe468fd5b280f183b0ac8b217aca050208baa2c2c9b02936a380"} Jul 07 17:48:51.856236 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.856128 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:51.875372 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:51.875255 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" podStartSLOduration=0.875232709 podStartE2EDuration="875.232709ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:48:51.875074241 +0000 UTC m=+2483.735154316" watchObservedRunningTime="2026-07-07 17:48:51.875232709 +0000 UTC m=+2483.735312784" Jul 07 17:48:52.404991 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:52.404955 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-8fdt6_45d4ecd5-0df8-4ea0-8cfd-20fb7fa2ebf9/download-server/0.log" Jul 07 17:48:54.213412 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:54.213382 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4jnln_7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b/dns/0.log" Jul 07 17:48:54.240136 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:54.240092 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4jnln_7d7385a2-6dfd-41d7-a6e6-2c05f8ce5c4b/kube-rbac-proxy/0.log" Jul 07 17:48:54.440147 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:54.440114 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tqtx4_3ff53b6a-4006-4d81-88c4-6cae9a6590ef/dns-node-resolver/0.log" Jul 07 17:48:55.152644 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:55.152608 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f9gq4_db8bf3da-6be2-460d-b95d-004822f9a091/node-ca/0.log" Jul 07 17:48:56.291547 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:56.291515 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45rdln_28d6ce2b-c497-4b33-b5a2-2447b66361f5/istio-proxy/0.log" Jul 07 17:48:56.443251 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:56.443221 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-4wv8v_3cb9abe8-8b1e-4b48-93ae-b043104a9c43/kube-auth-proxy/0.log" Jul 07 17:48:57.467245 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:57.467215 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbvmj_52dc686d-68c7-46e5-b4c0-8f622123e920/serve-healthcheck-canary/0.log" Jul 07 17:48:57.875075 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:57.875030 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-4rdr7" Jul 07 17:48:58.096033 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:58.096005 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-txwgh_28c670d9-ad00-4f66-a082-613c023982f5/kube-rbac-proxy/0.log" Jul 07 17:48:58.122222 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:58.122189 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-txwgh_28c670d9-ad00-4f66-a082-613c023982f5/exporter/0.log" Jul 07 17:48:58.145394 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:48:58.145322 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-txwgh_28c670d9-ad00-4f66-a082-613c023982f5/extractor/0.log" Jul 07 17:49:00.673258 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:00.673223 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z5btm_0b9b46ee-c52a-4b30-a95b-96c85e84c51b/manager/0.log" Jul 07 17:49:00.933242 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:00.933210 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zcmnd_61271ea3-07a6-4510-bca4-5bfdf8616073/manager/0.log" Jul 07 17:49:01.047717 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:01.047633 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724105-8mmq5_b7d928e9-4877-422a-833b-41685e722ba3/cleanup/0.log" Jul 07 17:49:01.399601 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:01.399548 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-ktth7_8119380d-de5a-46dd-a204-ca61cc80e57b/manager/0.log" Jul 07 17:49:01.438103 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:01.438074 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-ktth7_8119380d-de5a-46dd-a204-ca61cc80e57b/copy-manifests/0.log" Jul 07 17:49:01.665438 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:01.665406 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45rdln_28d6ce2b-c497-4b33-b5a2-2447b66361f5/istio-proxy/0.log" Jul 07 17:49:01.785874 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:01.785844 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-4wv8v_3cb9abe8-8b1e-4b48-93ae-b043104a9c43/kube-auth-proxy/0.log" Jul 07 17:49:02.994474 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:02.994438 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bbf97565b-2hm9v_139ed07a-0356-426c-bdb2-0e4c166a916c/manager/0.log" Jul 07 17:49:03.211783 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:03.211760 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45rdln_28d6ce2b-c497-4b33-b5a2-2447b66361f5/istio-proxy/0.log" Jul 07 17:49:03.326217 ip-10-0-133-16 kubenswrapper[2582]: I0707 17:49:03.326174 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-4wv8v_3cb9abe8-8b1e-4b48-93ae-b043104a9c43/kube-auth-proxy/0.log"