Jul 24 15:19:57.964936 ip-10-0-133-102 systemd[1]: Starting Kubernetes Kubelet... Jul 24 15:19:58.338054 ip-10-0-133-102 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 15:19:58.338054 ip-10-0-133-102 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 15:19:58.338054 ip-10-0-133-102 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 15:19:58.338054 ip-10-0-133-102 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 15:19:58.338054 ip-10-0-133-102 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 15:19:58.339780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.339659 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 15:19:58.344312 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344296 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:19:58.344312 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344312 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344316 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344319 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344323 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344326 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344329 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344331 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344334 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344337 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344340 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344342 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344345 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344347 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344350 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344352 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344355 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344358 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344360 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344363 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:19:58.344373 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344365 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344372 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344375 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344378 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344381 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344383 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344386 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344389 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344392 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344394 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344397 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344399 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344402 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344404 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344407 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344409 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344412 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344414 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344417 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344419 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:19:58.344854 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344422 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344427 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344431 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344433 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344436 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344438 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344440 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344443 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344445 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344447 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344450 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344452 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344454 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344457 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344460 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344463 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344466 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344468 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344471 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:19:58.345336 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344474 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344478 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344481 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344484 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344486 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344489 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344492 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344494 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344498 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344501 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344504 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344507 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344510 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344512 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344515 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344517 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344520 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344522 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344524 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344527 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:19:58.345926 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344529 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344532 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344534 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344537 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344539 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344541 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344544 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344547 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344550 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344553 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344555 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344558 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344560 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344563 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344565 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344567 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344570 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344572 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344575 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.344578 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:19:58.346702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345296 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345303 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345306 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345309 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345311 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345314 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345317 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345320 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345323 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345325 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345328 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345330 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345333 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345335 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345338 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345341 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345343 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345346 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345350 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345354 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345356 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:19:58.347269 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345360 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345362 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345365 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345368 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345371 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345373 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345376 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345378 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345381 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345384 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345387 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345389 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345391 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345394 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345397 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345399 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345402 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345405 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345408 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345410 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:19:58.347792 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345413 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345415 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345418 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345420 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345423 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345426 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345428 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345431 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345433 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345436 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345439 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345442 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345444 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345448 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345450 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345454 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345458 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345460 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345463 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:19:58.348276 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345465 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345468 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345470 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345473 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345475 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345478 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345480 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345483 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345485 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345487 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345490 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345492 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345495 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345497 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345500 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345503 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345505 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345508 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345510 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345513 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:19:58.348754 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345515 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345518 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345520 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345524 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345529 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345532 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345535 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345582 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345631 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345635 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345639 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345643 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345660 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345663 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345667 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345670 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345673 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345676 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.345679 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:19:58.349231 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347689 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347700 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347704 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347707 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347710 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347713 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347716 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347719 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347721 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347724 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347726 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347729 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347731 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347734 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347736 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347739 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347741 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347745 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347752 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:19:58.349767 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347755 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347758 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347761 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347764 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347766 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347769 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347772 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347774 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347777 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347780 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347782 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347785 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347787 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347790 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347793 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347796 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347798 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347801 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347803 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347806 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:19:58.350259 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347808 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347811 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347813 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347817 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347819 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347821 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347824 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347826 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347828 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347831 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347834 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347836 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347839 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347841 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347845 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347848 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347851 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347855 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347857 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:19:58.350757 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347860 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347862 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347865 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347867 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347869 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347872 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347875 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347878 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347881 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347883 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347885 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347889 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347892 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347895 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347897 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347900 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347902 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347904 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347907 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347909 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:19:58.351227 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347912 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347914 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347917 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347919 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347922 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347924 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347926 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347929 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347931 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347934 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347937 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347941 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347943 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347946 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347949 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347952 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347954 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347957 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347960 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:19:58.351763 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347962 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.347965 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348032 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348040 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348047 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348051 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348055 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348058 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348064 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348069 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348072 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348075 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348078 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348081 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348084 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348087 2564 flags.go:64] FLAG: --cgroup-root="" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348090 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348093 2564 flags.go:64] FLAG: --client-ca-file="" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348096 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348098 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348102 2564 flags.go:64] FLAG: --cluster-domain="" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348105 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348108 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348111 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 15:19:58.352228 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348114 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348117 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348120 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348123 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348126 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348129 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348132 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348135 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348138 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348141 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348145 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348148 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348151 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348153 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348156 2564 flags.go:64] FLAG: --enable-server="true" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348160 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348164 2564 flags.go:64] FLAG: --event-burst="100" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348168 2564 flags.go:64] FLAG: --event-qps="50" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348170 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348173 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348176 2564 flags.go:64] FLAG: --eviction-hard="" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348179 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348182 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348185 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348188 2564 flags.go:64] FLAG: --eviction-soft="" Jul 24 15:19:58.352833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348191 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348193 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348196 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348199 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348201 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348204 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348207 2564 flags.go:64] FLAG: --feature-gates="" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348215 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348218 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348221 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348224 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348227 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348230 2564 flags.go:64] FLAG: --help="false" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348236 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348239 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348241 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348244 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348247 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348251 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348254 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348256 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348259 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348262 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 15:19:58.353422 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348266 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348269 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348271 2564 flags.go:64] FLAG: --kube-reserved="" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348274 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348277 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348280 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348283 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348286 2564 flags.go:64] FLAG: --lock-file="" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348288 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348291 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348294 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348303 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348306 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348309 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348311 2564 flags.go:64] FLAG: --logging-format="text" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348314 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348317 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348320 2564 flags.go:64] FLAG: --manifest-url="" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348323 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348328 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348331 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348334 2564 flags.go:64] FLAG: --max-pods="110" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348338 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348341 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348344 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 15:19:58.354037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348347 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348350 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348352 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348355 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348361 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348364 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348367 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348370 2564 flags.go:64] FLAG: --pod-cidr="" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348374 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348379 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348382 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348385 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348388 2564 flags.go:64] FLAG: --port="10250" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348391 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348394 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d6a916c0173f4028" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348397 2564 flags.go:64] FLAG: --qos-reserved="" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348399 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348402 2564 flags.go:64] FLAG: --register-node="true" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348405 2564 flags.go:64] FLAG: --register-with-taints="" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348408 2564 flags.go:64] FLAG: --registry-burst="10" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348411 2564 flags.go:64] FLAG: --registry-qps="5" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348414 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348416 2564 flags.go:64] FLAG: --reserved-memory="" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348419 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348422 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 15:19:58.354655 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348427 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348430 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348433 2564 flags.go:64] FLAG: --runonce="false" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348436 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348439 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348442 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348445 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348447 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348450 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348453 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348456 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348458 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348461 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348464 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348467 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348470 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348473 2564 flags.go:64] FLAG: --system-cgroups="" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348476 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348481 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348484 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348487 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348490 2564 flags.go:64] FLAG: --tls-min-version="" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348493 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348496 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348498 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 15:19:58.355334 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348501 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348504 2564 flags.go:64] FLAG: --v="2" Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348507 2564 flags.go:64] FLAG: --version="false" Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348511 2564 flags.go:64] FLAG: --vmodule="" Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348514 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.348517 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348630 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348635 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348638 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348641 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348659 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348662 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348664 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348667 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348669 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348672 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348674 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348677 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348679 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348682 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348684 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:19:58.355954 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348686 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348689 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348691 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348694 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348697 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348699 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348702 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348704 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348707 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348709 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348711 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348714 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348716 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348719 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348721 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348724 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348726 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348729 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348733 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348735 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:19:58.356462 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348738 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348740 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348743 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348745 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348747 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348750 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348752 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348754 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348757 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348760 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348762 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348765 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348767 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348770 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348772 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348775 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348778 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348782 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348785 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:19:58.356963 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348788 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348791 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348793 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348796 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348799 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348801 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348803 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348806 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348810 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348813 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348816 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348821 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348823 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348826 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348829 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348831 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348834 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348836 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348839 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:19:58.357411 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348841 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348844 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348846 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348849 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348851 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348853 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348856 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348858 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348861 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348864 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348867 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348871 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348873 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348876 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348879 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348881 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348884 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348887 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348890 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348892 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:19:58.357879 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348895 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348898 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348900 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348903 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348908 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.348910 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.349477 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.356445 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.356461 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.357222 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.357234 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357305 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357310 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357313 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357316 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:19:58.358398 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357319 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357322 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357324 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357327 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357329 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357332 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357334 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357337 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357340 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357342 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357344 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357347 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357350 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357352 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357355 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357357 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357359 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357362 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357364 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357367 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:19:58.358796 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357370 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357372 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357374 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357377 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357379 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357383 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357386 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357389 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357392 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357394 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357397 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357399 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357402 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357405 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357408 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357412 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357414 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357417 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357420 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357422 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:19:58.359295 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357425 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357427 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357430 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357433 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357435 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357438 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357441 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357445 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357448 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357451 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357453 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357456 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357458 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357461 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357464 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357467 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357469 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357473 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357475 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:19:58.359770 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357478 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357480 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357483 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357485 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357488 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357490 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357493 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357495 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357498 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357500 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357503 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357505 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357508 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357510 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357512 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357515 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357517 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357520 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357522 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357525 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:19:58.360224 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357527 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357530 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357532 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357535 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357537 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357539 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357542 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357545 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357549 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357552 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357555 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357558 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357562 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357564 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357567 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:19:58.360702 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357569 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.357574 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357698 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357703 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357706 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357709 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357712 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357715 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357717 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357720 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357722 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357725 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357727 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357729 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357732 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:19:58.361072 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357734 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357737 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357739 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357742 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357744 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357746 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357749 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357752 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357755 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357758 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357760 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357763 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357766 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357769 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357773 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357776 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357779 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357782 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357784 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:19:58.361429 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357787 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357789 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357792 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357794 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357797 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357799 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357801 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357804 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357806 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357808 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357811 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357813 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357816 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357818 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357821 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357823 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357825 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357828 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357832 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:19:58.361907 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357835 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357838 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357841 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357844 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357847 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357849 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357852 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357854 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357857 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357859 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357862 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357865 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357867 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357869 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357872 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357874 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357877 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357879 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357882 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357885 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:19:58.362438 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357887 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357890 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357893 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357895 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357897 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357900 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357902 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357905 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357907 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357909 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357912 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357922 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357926 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357928 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357930 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357933 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357936 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357938 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357941 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357943 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:19:58.362942 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357946 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357948 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357951 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357973 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357976 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357978 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357981 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:58.357983 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.357988 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.358124 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.361845 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.363251 2564 server.go:1019] "Starting client certificate rotation" Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.363353 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 15:19:58.363442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.363390 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 15:19:58.365889 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.365876 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 15:19:58.367141 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.367127 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 15:19:58.376221 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.376198 2564 log.go:25] "Validated CRI v1 runtime API" Jul 24 15:19:58.380180 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.380165 2564 log.go:25] "Validated CRI v1 image API" Jul 24 15:19:58.380898 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.380879 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 15:19:58.383399 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.383376 2564 fs.go:135] Filesystem UUIDs: map[2665a08a-7ab4-4a14-83c4-edccc2d17f79:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eecc0b59-5207-46ed-a742-098b3f632f35:/dev/nvme0n1p3] Jul 24 15:19:58.383461 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.383396 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 15:19:58.388488 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.388377 2564 manager.go:216] Machine: {Timestamp:2026-07-24 15:19:58.387065263 +0000 UTC m=+0.310990583 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3140103 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2797fdb474cc82c007fdb5ab226021 SystemUUID:ec2797fd-b474-cc82-c007-fdb5ab226021 BootID:beeb2180-b205-44e0-9a0c-e95bc592a819 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:6c:c7:34:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:6c:c7:34:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:53:20:fc:2f:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 15:19:58.388488 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.388482 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 15:19:58.388620 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.388567 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 15:19:58.391254 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.391225 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 15:19:58.391414 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.391256 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 15:19:58.391586 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.391571 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 15:19:58.391978 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.391965 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 15:19:58.392029 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.391984 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 15:19:58.392029 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.391995 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 15:19:58.392670 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.392641 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 15:19:58.393923 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.393904 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 15:19:58.396804 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.396792 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 24 15:19:58.396941 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.396930 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 15:19:58.397006 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.396969 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 24 15:19:58.397006 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.396983 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 15:19:58.397085 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.397006 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 15:19:58.397085 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.397020 2564 kubelet.go:398] "Adding apiserver pod source" Jul 24 15:19:58.397085 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.397034 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 15:19:58.397406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.397391 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 15:19:58.397465 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.397414 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 15:19:58.399522 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.399496 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 15:19:58.400852 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.400835 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 15:19:58.400929 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.400875 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 15:19:58.401475 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401462 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 15:19:58.401543 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401481 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 15:19:58.401543 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401490 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 15:19:58.401543 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401499 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 15:19:58.401543 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401512 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 15:19:58.401543 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401521 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 15:19:58.401750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401568 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 15:19:58.401750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401578 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 15:19:58.401750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401589 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 15:19:58.401750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401597 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 15:19:58.401750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401610 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 15:19:58.401750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.401623 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 15:19:58.402432 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.402421 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 15:19:58.402480 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.402435 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 15:19:58.405639 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.405623 2564 server.go:1268] "Started kubelet" Jul 24 15:19:58.405742 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.405720 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 15:19:58.405908 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.405718 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 15:19:58.406122 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.406102 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 15:19:58.406547 ip-10-0-133-102 systemd[1]: Started Kubernetes Kubelet. Jul 24 15:19:58.411812 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.411783 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 15:19:58.413336 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.413295 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 15:19:58.413635 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.413620 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 24 15:19:58.415374 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.415356 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 15:19:58.415374 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.415367 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjv72" Jul 24 15:19:58.416148 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.415987 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 15:19:58.416148 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.415988 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 15:19:58.416310 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.415420 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-102.ec2.internal.18c542e99a49938e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-102.ec2.internal,UID:ip-10-0-133-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-102.ec2.internal,},FirstTimestamp:2026-07-24 15:19:58.405596046 +0000 UTC m=+0.329521345,LastTimestamp:2026-07-24 15:19:58.405596046 +0000 UTC m=+0.329521345,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-102.ec2.internal,}" Jul 24 15:19:58.422785 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.422768 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjv72" Jul 24 15:19:58.423901 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.423884 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 15:19:58.424430 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.424415 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 15:19:58.424766 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.424734 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.425183 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425165 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 15:19:58.425265 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425186 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 15:19:58.425265 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425165 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 15:19:58.425355 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425278 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 24 15:19:58.425355 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425288 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 24 15:19:58.425613 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425598 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 15:19:58.425674 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425616 2564 factory.go:55] Registering systemd factory Jul 24 15:19:58.425674 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425624 2564 factory.go:223] Registration of the systemd container factory successfully Jul 24 15:19:58.425889 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425872 2564 factory.go:153] Registering CRI-O factory Jul 24 15:19:58.426014 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425894 2564 factory.go:223] Registration of the crio container factory successfully Jul 24 15:19:58.426014 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425926 2564 factory.go:103] Registering Raw factory Jul 24 15:19:58.426014 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.425948 2564 manager.go:1196] Started watching for new ooms in manager Jul 24 15:19:58.426477 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.426451 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 15:19:58.426548 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.426481 2564 manager.go:349] Starting recovery of all containers Jul 24 15:19:58.426548 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.426517 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 15:19:58.432529 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.432482 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 15:19:58.437240 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.437130 2564 manager.go:354] Recovery completed Jul 24 15:19:58.441593 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.441575 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.443454 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.443440 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.443515 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.443470 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.443515 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.443480 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.444175 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444161 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 15:19:58.444175 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444173 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 15:19:58.444272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444188 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 24 15:19:58.444272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444221 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.444893 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444881 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.444946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444905 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.444946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.444916 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.445606 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.445595 2564 policy_none.go:49] "None policy: Start" Jul 24 15:19:58.445638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.445610 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 15:19:58.445638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.445620 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 15:19:58.446896 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.446885 2564 policy_none.go:47] "Start" Jul 24 15:19:58.489173 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489154 2564 manager.go:337] "Starting Device Plugin manager" Jul 24 15:19:58.489274 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.489192 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 15:19:58.489274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489201 2564 server.go:87] "Starting device plugin registration server" Jul 24 15:19:58.489434 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489421 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 15:19:58.489466 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489438 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 15:19:58.489557 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489534 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 15:19:58.489679 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489632 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 15:19:58.489679 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.489641 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 15:19:58.490233 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.490203 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 15:19:58.490297 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.490263 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.527393 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.527363 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 15:19:58.530272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.527421 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 15:19:58.530272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.527452 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 15:19:58.530272 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.527493 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 15:19:58.530272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.529924 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:19:58.589613 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.589538 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.590499 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.590481 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.590592 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.590514 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.590592 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.590526 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.590592 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.590552 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.598605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.598586 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.598718 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.598614 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-102.ec2.internal\": node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.613446 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.613416 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.627785 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.627757 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal"] Jul 24 15:19:58.627893 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.627823 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.628795 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.628780 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.628887 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.628807 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.628887 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.628820 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.630389 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.630375 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.630519 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.630505 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.630582 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.630555 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.631983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.631335 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.631983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.631336 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.631983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.631385 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.631983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.631398 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.631983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.631364 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.631983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.631447 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.632914 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.632898 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.632997 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.632933 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:19:58.633631 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.633616 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:19:58.633727 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.633666 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:19:58.633727 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.633677 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:19:58.654383 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.654356 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-102.ec2.internal\" not found" node="ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.658417 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.658399 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-102.ec2.internal\" not found" node="ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.714059 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.714025 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.814953 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.814922 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.826198 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.826178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f69636379af9c8864e9742b149d3b2df-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal\" (UID: \"f69636379af9c8864e9742b149d3b2df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.826280 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.826203 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f69636379af9c8864e9742b149d3b2df-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal\" (UID: \"f69636379af9c8864e9742b149d3b2df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.826280 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.826218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/414259330d98a4b19eb5aa9f7c16c705-config\") pod \"kube-apiserver-proxy-ip-10-0-133-102.ec2.internal\" (UID: \"414259330d98a4b19eb5aa9f7c16c705\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.915697 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:58.915602 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:58.926941 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.926908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f69636379af9c8864e9742b149d3b2df-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal\" (UID: \"f69636379af9c8864e9742b149d3b2df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.927053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.926949 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f69636379af9c8864e9742b149d3b2df-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal\" (UID: \"f69636379af9c8864e9742b149d3b2df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.927053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.927009 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f69636379af9c8864e9742b149d3b2df-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal\" (UID: \"f69636379af9c8864e9742b149d3b2df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.927053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.927013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f69636379af9c8864e9742b149d3b2df-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal\" (UID: \"f69636379af9c8864e9742b149d3b2df\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.927174 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.927061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/414259330d98a4b19eb5aa9f7c16c705-config\") pod \"kube-apiserver-proxy-ip-10-0-133-102.ec2.internal\" (UID: \"414259330d98a4b19eb5aa9f7c16c705\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.927174 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.927096 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/414259330d98a4b19eb5aa9f7c16c705-config\") pod \"kube-apiserver-proxy-ip-10-0-133-102.ec2.internal\" (UID: \"414259330d98a4b19eb5aa9f7c16c705\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.958076 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.958035 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:58.961303 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:58.961282 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" Jul 24 15:19:59.015748 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:59.015713 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:59.116291 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:59.116268 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:59.216843 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:59.216768 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:59.317352 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:59.317324 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:59.362815 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.362795 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 15:19:59.363236 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.362956 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 15:19:59.418014 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:19:59.417985 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-102.ec2.internal\" not found" Jul 24 15:19:59.426559 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.426524 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 15:14:58 +0000 UTC" deadline="2028-05-05 15:29:10.645483288 +0000 UTC" Jul 24 15:19:59.426559 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.426558 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15624h9m11.21893018s" Jul 24 15:19:59.458249 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:59.458217 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414259330d98a4b19eb5aa9f7c16c705.slice/crio-b749bc18654a27739cb0a89a3de2fb65ef9a87c27e89df5388f7cabfc7711ec6 WatchSource:0}: Error finding container b749bc18654a27739cb0a89a3de2fb65ef9a87c27e89df5388f7cabfc7711ec6: Status 404 returned error can't find the container with id b749bc18654a27739cb0a89a3de2fb65ef9a87c27e89df5388f7cabfc7711ec6 Jul 24 15:19:59.458723 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:19:59.458684 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69636379af9c8864e9742b149d3b2df.slice/crio-1cca5e067e4cbc462b6e155efa8f310681138e2b84d8ffc47ae38a40c3d70dd4 WatchSource:0}: Error finding container 1cca5e067e4cbc462b6e155efa8f310681138e2b84d8ffc47ae38a40c3d70dd4: Status 404 returned error can't find the container with id 1cca5e067e4cbc462b6e155efa8f310681138e2b84d8ffc47ae38a40c3d70dd4 Jul 24 15:19:59.465313 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.465278 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:19:59.502927 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.502895 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:19:59.525571 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.525545 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" Jul 24 15:19:59.531147 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.531103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" event={"ID":"f69636379af9c8864e9742b149d3b2df","Type":"ContainerStarted","Data":"1cca5e067e4cbc462b6e155efa8f310681138e2b84d8ffc47ae38a40c3d70dd4"} Jul 24 15:19:59.532082 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.532060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" event={"ID":"414259330d98a4b19eb5aa9f7c16c705","Type":"ContainerStarted","Data":"b749bc18654a27739cb0a89a3de2fb65ef9a87c27e89df5388f7cabfc7711ec6"} Jul 24 15:19:59.534492 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.534475 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 15:19:59.536270 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.536256 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" Jul 24 15:19:59.547394 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.547373 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 15:19:59.742601 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.742511 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:19:59.892373 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:19:59.892337 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:20:00.398564 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.398536 2564 apiserver.go:52] "Watching apiserver" Jul 24 15:20:00.408805 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.408770 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 15:20:00.410030 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.410000 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-8ztqv","kube-system/global-pull-secret-syncer-24625","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh","openshift-cluster-node-tuning-operator/tuned-9g2rg","openshift-dns/node-resolver-tn7vd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal","openshift-network-diagnostics/network-check-target-4sbjk","kube-system/konnectivity-agent-cwk2v","kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal","openshift-image-registry/node-ca-dhhwh","openshift-multus/multus-additional-cni-plugins-9fjwh","openshift-multus/multus-rk7tm","openshift-multus/network-metrics-daemon-9v4nd","openshift-network-operator/iptables-alerter-8jk4r"] Jul 24 15:20:00.412048 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.412021 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:00.412167 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.412097 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:00.413989 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.413966 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:00.414120 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.414055 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:00.415541 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.415506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.417240 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.417219 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.418780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.418238 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.418780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.418389 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dfqdb\"" Jul 24 15:20:00.418780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.418462 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 15:20:00.418780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.418540 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.419584 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.419560 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.420135 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.420114 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pfpdm\"" Jul 24 15:20:00.420231 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.420157 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.420293 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.420244 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.421197 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.421177 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.422441 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.422208 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.422441 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.422246 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.422441 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.422208 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2sqkz\"" Jul 24 15:20:00.423265 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.422927 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.423821 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.423802 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 15:20:00.423901 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.423829 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.426120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.424155 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.426120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.424333 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 15:20:00.426120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.424829 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dgrgr\"" Jul 24 15:20:00.426120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.425278 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.426120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.425668 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tbrg2\"" Jul 24 15:20:00.426120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.425748 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 15:20:00.426773 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.426177 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 15:20:00.427453 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.427430 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.427564 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.427541 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:00.428329 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.428302 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.429127 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.429196 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.429615 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.430107 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.430804 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.431019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.433274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.431516 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-96wll\"" Jul 24 15:20:00.433545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.433516 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.433952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.433841 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.435303 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.435279 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.435487 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.435470 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.435288 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8bsg9\"" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436397 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd9f892d-661f-4774-8a30-749cf1235869-etc-tuned\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436439 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436487 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-etc-selinux\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436529 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysctl-conf\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-sys\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.436632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-log-socket\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436649 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-env-overrides\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-slash\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-ovn\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mqhc\" (UniqueName: \"kubernetes.io/projected/6755c18e-292e-4b86-b685-41bd508fdcfc-kube-api-access-6mqhc\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436866 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d8711f9-2375-40fb-8a92-ba56c25afc17-dbus\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436900 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-device-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d9172f6-f70c-46db-8607-b3413f6afb8d-hosts-file\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.436968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-run\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437000 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-cni-bin\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-ovnkube-config\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95805928-870f-43ab-ba4a-7d86e2e7836f-host-slash\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-modprobe-d\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.437162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437158 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysctl-d\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xm58\" (UniqueName: \"kubernetes.io/projected/95805928-870f-43ab-ba4a-7d86e2e7836f-kube-api-access-5xm58\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437244 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437293 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-systemd\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6755c18e-292e-4b86-b685-41bd508fdcfc-ovn-node-metrics-cert\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd9f892d-661f-4774-8a30-749cf1235869-tmp\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-run-netns\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437467 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-systemd\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-var-lib-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-socket-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.437850 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.437617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-sys-fs\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.438522 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p4fm2\"" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.438711 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.438890 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439057 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439221 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-cni-netd\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-var-lib-kubelet\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-etc-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439383 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439407 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d8711f9-2375-40fb-8a92-ba56c25afc17-kubelet-config\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-registration-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzx5\" (UniqueName: \"kubernetes.io/projected/8213e2c5-8b95-406f-b783-5616c32452a9-kube-api-access-4vzx5\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439498 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysconfig\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439521 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-lib-modules\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4nv\" (UniqueName: \"kubernetes.io/projected/cd9f892d-661f-4774-8a30-749cf1235869-kube-api-access-7v4nv\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-systemd-units\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d9172f6-f70c-46db-8607-b3413f6afb8d-tmp-dir\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdwj6\" (UniqueName: \"kubernetes.io/projected/9d9172f6-f70c-46db-8607-b3413f6afb8d-kube-api-access-hdwj6\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439654 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 15:20:00.439758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439668 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-host\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439691 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-node-log\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95805928-870f-43ab-ba4a-7d86e2e7836f-iptables-alerter-script\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drqkr\" (UniqueName: \"kubernetes.io/projected/ac7f3760-5de1-4c0b-8838-f76677aa4705-kube-api-access-drqkr\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439804 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3313863b-64c5-4ddd-b6d4-bd1401402c52-agent-certs\") pod \"konnectivity-agent-cwk2v\" (UID: \"3313863b-64c5-4ddd-b6d4-bd1401402c52\") " pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3313863b-64c5-4ddd-b6d4-bd1401402c52-konnectivity-ca\") pod \"konnectivity-agent-cwk2v\" (UID: \"3313863b-64c5-4ddd-b6d4-bd1401402c52\") " pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-kubelet\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439875 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-ovnkube-script-lib\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439910 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.439953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-kubernetes\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.440917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.440068 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.443382 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.443162 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 15:20:00.443382 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.443254 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gwmmq\"" Jul 24 15:20:00.461178 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.460902 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5nbp7" Jul 24 15:20:00.472709 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.472414 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5nbp7" Jul 24 15:20:00.526516 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.526483 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 15:20:00.540421 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540389 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd9f892d-661f-4774-8a30-749cf1235869-etc-tuned\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.540604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540438 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-daemon-config\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.540604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-etc-selinux\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.540604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-host\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.540604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540552 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysctl-conf\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.540604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540575 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-sys\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-log-socket\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-env-overrides\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-slash\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540771 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-ovn\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqhc\" (UniqueName: \"kubernetes.io/projected/6755c18e-292e-4b86-b685-41bd508fdcfc-kube-api-access-6mqhc\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d8711f9-2375-40fb-8a92-ba56c25afc17-dbus\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-os-release\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-cnibin\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540930 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-netns\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.540979 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540955 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-device-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.540990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-serviceca\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqggc\" (UniqueName: \"kubernetes.io/projected/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-kube-api-access-xqggc\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541043 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzcdt\" (UniqueName: \"kubernetes.io/projected/f7ca19f5-babd-47c0-a92b-be5d34696cd6-kube-api-access-lzcdt\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-socket-dir-parent\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-k8s-cni-cncf-io\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-multus-certs\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d9172f6-f70c-46db-8607-b3413f6afb8d-hosts-file\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-run\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-cni-bin\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-ovnkube-config\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541276 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541302 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-cni-bin\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541329 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95805928-870f-43ab-ba4a-7d86e2e7836f-host-slash\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-modprobe-d\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541381 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysctl-d\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xm58\" (UniqueName: \"kubernetes.io/projected/95805928-870f-43ab-ba4a-7d86e2e7836f-kube-api-access-5xm58\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.541581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-systemd\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6755c18e-292e-4b86-b685-41bd508fdcfc-ovn-node-metrics-cert\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-cni-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541573 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-hostroot\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-conf-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd9f892d-661f-4774-8a30-749cf1235869-tmp\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541650 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-run-netns\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541683 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-systemd\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541712 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-var-lib-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541767 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541798 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-socket-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-sys-fs\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541855 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-cni-netd\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cnibin\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aadb56a-0ce2-48d2-b3f1-e874409cd867-cni-binary-copy\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.542448 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541963 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-var-lib-kubelet\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.541994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-etc-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542018 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542092 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d8711f9-2375-40fb-8a92-ba56c25afc17-kubelet-config\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d9172f6-f70c-46db-8607-b3413f6afb8d-hosts-file\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-run\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542285 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-cni-bin\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542022 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d8711f9-2375-40fb-8a92-ba56c25afc17-kubelet-config\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-os-release\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-registration-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzx5\" (UniqueName: \"kubernetes.io/projected/8213e2c5-8b95-406f-b783-5616c32452a9-kube-api-access-4vzx5\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysconfig\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-lib-modules\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4nv\" (UniqueName: \"kubernetes.io/projected/cd9f892d-661f-4774-8a30-749cf1235869-kube-api-access-7v4nv\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-systemd-units\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-system-cni-dir\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.543271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-cni-multus\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542899 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-ovnkube-config\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d9172f6-f70c-46db-8607-b3413f6afb8d-tmp-dir\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.542974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdwj6\" (UniqueName: \"kubernetes.io/projected/9d9172f6-f70c-46db-8607-b3413f6afb8d-kube-api-access-hdwj6\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-host\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-node-log\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-system-cni-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-etc-kubernetes\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95805928-870f-43ab-ba4a-7d86e2e7836f-iptables-alerter-script\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drqkr\" (UniqueName: \"kubernetes.io/projected/ac7f3760-5de1-4c0b-8838-f76677aa4705-kube-api-access-drqkr\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543173 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543177 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3313863b-64c5-4ddd-b6d4-bd1401402c52-agent-certs\") pod \"konnectivity-agent-cwk2v\" (UID: \"3313863b-64c5-4ddd-b6d4-bd1401402c52\") " pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543233 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3313863b-64c5-4ddd-b6d4-bd1401402c52-konnectivity-ca\") pod \"konnectivity-agent-cwk2v\" (UID: \"3313863b-64c5-4ddd-b6d4-bd1401402c52\") " pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543257 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95805928-870f-43ab-ba4a-7d86e2e7836f-host-slash\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-kubelet\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-ovnkube-script-lib\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543303 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-device-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.544143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543330 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543347 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d9172f6-f70c-46db-8607-b3413f6afb8d-tmp-dir\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543483 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-sys\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543523 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543531 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysctl-conf\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-kubernetes\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-kubelet\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.543648 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543669 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnsxj\" (UniqueName: \"kubernetes.io/projected/8aadb56a-0ce2-48d2-b3f1-e874409cd867-kube-api-access-gnsxj\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.543724 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:01.04370544 +0000 UTC m=+2.967630727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543790 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-log-socket\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543793 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-etc-selinux\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.543931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-modprobe-d\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-socket-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544154 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-sys-fs\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544332 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-slash\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545070 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544337 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysctl-d\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-cni-netd\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-registration-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544431 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-etc-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544437 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac7f3760-5de1-4c0b-8838-f76677aa4705-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-sysconfig\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544491 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-host\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544515 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-node-log\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.544912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-env-overrides\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545022 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-lib-modules\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95805928-870f-43ab-ba4a-7d86e2e7836f-iptables-alerter-script\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545361 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-kubernetes\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545414 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-var-lib-openvswitch\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-etc-systemd\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545629 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd9f892d-661f-4774-8a30-749cf1235869-var-lib-kubelet\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.545762 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.545817 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:01.045799954 +0000 UTC m=+2.969725262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:00.545963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545861 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-systemd-units\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545899 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-kubelet\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3313863b-64c5-4ddd-b6d4-bd1401402c52-konnectivity-ca\") pod \"konnectivity-agent-cwk2v\" (UID: \"3313863b-64c5-4ddd-b6d4-bd1401402c52\") " pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd9f892d-661f-4774-8a30-749cf1235869-etc-tuned\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.545995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d8711f9-2375-40fb-8a92-ba56c25afc17-dbus\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.546031 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-ovn\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.546036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-run-systemd\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.546060 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3313863b-64c5-4ddd-b6d4-bd1401402c52-agent-certs\") pod \"konnectivity-agent-cwk2v\" (UID: \"3313863b-64c5-4ddd-b6d4-bd1401402c52\") " pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.546077 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.546789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.546063 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6755c18e-292e-4b86-b685-41bd508fdcfc-host-run-netns\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.547225 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.547020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6755c18e-292e-4b86-b685-41bd508fdcfc-ovnkube-script-lib\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.548970 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.548946 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6755c18e-292e-4b86-b685-41bd508fdcfc-ovn-node-metrics-cert\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.549623 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.549599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd9f892d-661f-4774-8a30-749cf1235869-tmp\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.557809 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.557787 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:00.557913 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.557815 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:00.557913 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.557829 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:00.558457 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:00.557905 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:01.057884134 +0000 UTC m=+2.981809423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:00.558664 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.558503 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:20:00.560698 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.560640 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drqkr\" (UniqueName: \"kubernetes.io/projected/ac7f3760-5de1-4c0b-8838-f76677aa4705-kube-api-access-drqkr\") pod \"aws-ebs-csi-driver-node-8nldh\" (UID: \"ac7f3760-5de1-4c0b-8838-f76677aa4705\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.560942 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.560919 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xm58\" (UniqueName: \"kubernetes.io/projected/95805928-870f-43ab-ba4a-7d86e2e7836f-kube-api-access-5xm58\") pod \"iptables-alerter-8jk4r\" (UID: \"95805928-870f-43ab-ba4a-7d86e2e7836f\") " pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.561022 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.560917 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdwj6\" (UniqueName: \"kubernetes.io/projected/9d9172f6-f70c-46db-8607-b3413f6afb8d-kube-api-access-hdwj6\") pod \"node-resolver-tn7vd\" (UID: \"9d9172f6-f70c-46db-8607-b3413f6afb8d\") " pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.561022 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.560959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mqhc\" (UniqueName: \"kubernetes.io/projected/6755c18e-292e-4b86-b685-41bd508fdcfc-kube-api-access-6mqhc\") pod \"ovnkube-node-8ztqv\" (UID: \"6755c18e-292e-4b86-b685-41bd508fdcfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.561308 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.561279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4nv\" (UniqueName: \"kubernetes.io/projected/cd9f892d-661f-4774-8a30-749cf1235869-kube-api-access-7v4nv\") pod \"tuned-9g2rg\" (UID: \"cd9f892d-661f-4774-8a30-749cf1235869\") " pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.562622 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.562599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzx5\" (UniqueName: \"kubernetes.io/projected/8213e2c5-8b95-406f-b783-5616c32452a9-kube-api-access-4vzx5\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:00.644450 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644381 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnsxj\" (UniqueName: \"kubernetes.io/projected/8aadb56a-0ce2-48d2-b3f1-e874409cd867-kube-api-access-gnsxj\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.644607 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-daemon-config\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.644607 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-host\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.644607 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.644825 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644604 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-host\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.644825 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-os-release\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.644825 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-cnibin\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.644825 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644799 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-cnibin\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.644825 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644814 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-os-release\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644835 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-netns\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-serviceca\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.645073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644934 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqggc\" (UniqueName: \"kubernetes.io/projected/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-kube-api-access-xqggc\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.645073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644942 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-netns\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.644961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzcdt\" (UniqueName: \"kubernetes.io/projected/f7ca19f5-babd-47c0-a92b-be5d34696cd6-kube-api-access-lzcdt\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-socket-dir-parent\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645180 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-k8s-cni-cncf-io\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645188 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-daemon-config\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-multus-certs\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645240 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-cni-bin\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-socket-dir-parent\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645301 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-cni-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645309 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-cni-bin\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645345 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-serviceca\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.645369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645358 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-multus-certs\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645374 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-run-k8s-cni-cncf-io\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645436 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-hostroot\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-hostroot\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645422 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-cni-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-conf-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-multus-conf-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cnibin\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aadb56a-0ce2-48d2-b3f1-e874409cd867-cni-binary-copy\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645616 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-os-release\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645645 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cnibin\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-os-release\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-system-cni-dir\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645756 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645782 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-cni-multus\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-system-cni-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-etc-kubernetes\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.645895 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645910 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-kubelet\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645922 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-kubelet\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.646006 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-system-cni-dir\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.646013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-etc-kubernetes\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.646052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8aadb56a-0ce2-48d2-b3f1-e874409cd867-host-var-lib-cni-multus\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.645779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7ca19f5-babd-47c0-a92b-be5d34696cd6-system-cni-dir\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.646143 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aadb56a-0ce2-48d2-b3f1-e874409cd867-cni-binary-copy\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.646235 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.646497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.646431 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7ca19f5-babd-47c0-a92b-be5d34696cd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.657053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.656985 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnsxj\" (UniqueName: \"kubernetes.io/projected/8aadb56a-0ce2-48d2-b3f1-e874409cd867-kube-api-access-gnsxj\") pod \"multus-rk7tm\" (UID: \"8aadb56a-0ce2-48d2-b3f1-e874409cd867\") " pod="openshift-multus/multus-rk7tm" Jul 24 15:20:00.657053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.656999 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzcdt\" (UniqueName: \"kubernetes.io/projected/f7ca19f5-babd-47c0-a92b-be5d34696cd6-kube-api-access-lzcdt\") pod \"multus-additional-cni-plugins-9fjwh\" (UID: \"f7ca19f5-babd-47c0-a92b-be5d34696cd6\") " pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.657053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.657035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqggc\" (UniqueName: \"kubernetes.io/projected/d2699c8d-2cd9-44e3-9ada-1885e5bc0a47-kube-api-access-xqggc\") pod \"node-ca-dhhwh\" (UID: \"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47\") " pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.731298 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.731262 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" Jul 24 15:20:00.741785 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.741761 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" Jul 24 15:20:00.753061 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.753032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tn7vd" Jul 24 15:20:00.760086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.760063 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8jk4r" Jul 24 15:20:00.767316 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.767294 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:00.775536 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.775512 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:00.787253 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.787231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dhhwh" Jul 24 15:20:00.795466 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.795427 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" Jul 24 15:20:00.801263 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:00.801242 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rk7tm" Jul 24 15:20:01.049928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.049838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:01.049928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.049881 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:01.050164 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.050018 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:01.050164 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.050019 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:01.050164 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.050103 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:02.050075867 +0000 UTC m=+3.974001155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:01.050164 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.050124 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:02.050113334 +0000 UTC m=+3.974038625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:01.151340 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.151300 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:01.151547 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.151514 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:01.151607 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.151550 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:01.151607 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.151567 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:01.151750 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.151632 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:02.151612607 +0000 UTC m=+4.075537900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:01.475315 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.475268 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:15:00 +0000 UTC" deadline="2028-03-16 07:14:14.284030338 +0000 UTC" Jul 24 15:20:01.475315 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.475315 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14415h54m12.808734226s" Jul 24 15:20:01.513189 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.513155 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9172f6_f70c_46db_8607_b3413f6afb8d.slice/crio-32a1e71e529199aa49ef9cdba9348159d86296b60b5e39d4037078f214ac9416 WatchSource:0}: Error finding container 32a1e71e529199aa49ef9cdba9348159d86296b60b5e39d4037078f214ac9416: Status 404 returned error can't find the container with id 32a1e71e529199aa49ef9cdba9348159d86296b60b5e39d4037078f214ac9416 Jul 24 15:20:01.528168 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.527657 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:01.528168 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:01.527810 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:01.534201 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.534174 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95805928_870f_43ab_ba4a_7d86e2e7836f.slice/crio-bbf396e45a3973af1b05d88a1d401ae82ecacf85509788f01f76b8f3450ca3e3 WatchSource:0}: Error finding container bbf396e45a3973af1b05d88a1d401ae82ecacf85509788f01f76b8f3450ca3e3: Status 404 returned error can't find the container with id bbf396e45a3973af1b05d88a1d401ae82ecacf85509788f01f76b8f3450ca3e3 Jul 24 15:20:01.536382 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.536353 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3313863b_64c5_4ddd_b6d4_bd1401402c52.slice/crio-c17ba450582643ea9cfa579177b129b0d42bd82d5024ce84aee00c9483771379 WatchSource:0}: Error finding container c17ba450582643ea9cfa579177b129b0d42bd82d5024ce84aee00c9483771379: Status 404 returned error can't find the container with id c17ba450582643ea9cfa579177b129b0d42bd82d5024ce84aee00c9483771379 Jul 24 15:20:01.537308 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.537284 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aadb56a_0ce2_48d2_b3f1_e874409cd867.slice/crio-a22066232612763a034ba5c194921450c242bf4ca4e3991ab30faac635ddceb6 WatchSource:0}: Error finding container a22066232612763a034ba5c194921450c242bf4ca4e3991ab30faac635ddceb6: Status 404 returned error can't find the container with id a22066232612763a034ba5c194921450c242bf4ca4e3991ab30faac635ddceb6 Jul 24 15:20:01.537754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.537718 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8jk4r" event={"ID":"95805928-870f-43ab-ba4a-7d86e2e7836f","Type":"ContainerStarted","Data":"bbf396e45a3973af1b05d88a1d401ae82ecacf85509788f01f76b8f3450ca3e3"} Jul 24 15:20:01.539000 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.538908 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ca19f5_babd_47c0_a92b_be5d34696cd6.slice/crio-16dbb54831747d1f546234c604c8353511d15d1dbe1bfec02c12821226ad292d WatchSource:0}: Error finding container 16dbb54831747d1f546234c604c8353511d15d1dbe1bfec02c12821226ad292d: Status 404 returned error can't find the container with id 16dbb54831747d1f546234c604c8353511d15d1dbe1bfec02c12821226ad292d Jul 24 15:20:01.539528 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:01.539505 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tn7vd" event={"ID":"9d9172f6-f70c-46db-8607-b3413f6afb8d","Type":"ContainerStarted","Data":"32a1e71e529199aa49ef9cdba9348159d86296b60b5e39d4037078f214ac9416"} Jul 24 15:20:01.542944 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.542710 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2699c8d_2cd9_44e3_9ada_1885e5bc0a47.slice/crio-7b3b8d9998b15a6f8a56b503f7b5599eb6c8bac4280e301b335da4ef4b428d73 WatchSource:0}: Error finding container 7b3b8d9998b15a6f8a56b503f7b5599eb6c8bac4280e301b335da4ef4b428d73: Status 404 returned error can't find the container with id 7b3b8d9998b15a6f8a56b503f7b5599eb6c8bac4280e301b335da4ef4b428d73 Jul 24 15:20:01.543972 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.543944 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9f892d_661f_4774_8a30_749cf1235869.slice/crio-d7e02fcc847f2f3114e52ce70373447333952ce33aa8f05d0ed097facccf7d9e WatchSource:0}: Error finding container d7e02fcc847f2f3114e52ce70373447333952ce33aa8f05d0ed097facccf7d9e: Status 404 returned error can't find the container with id d7e02fcc847f2f3114e52ce70373447333952ce33aa8f05d0ed097facccf7d9e Jul 24 15:20:01.545106 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.545055 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6755c18e_292e_4b86_b685_41bd508fdcfc.slice/crio-bf96f1f5afcfbaaa847b101c3665215e561300731676d5e2fa3ce8b4a8595535 WatchSource:0}: Error finding container bf96f1f5afcfbaaa847b101c3665215e561300731676d5e2fa3ce8b4a8595535: Status 404 returned error can't find the container with id bf96f1f5afcfbaaa847b101c3665215e561300731676d5e2fa3ce8b4a8595535 Jul 24 15:20:01.547802 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:01.547779 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7f3760_5de1_4c0b_8838_f76677aa4705.slice/crio-f9501bdfac0de0d004209ec0ddd3f76a0ff3772712a5e2cfb2a943bfa7ea8500 WatchSource:0}: Error finding container f9501bdfac0de0d004209ec0ddd3f76a0ff3772712a5e2cfb2a943bfa7ea8500: Status 404 returned error can't find the container with id f9501bdfac0de0d004209ec0ddd3f76a0ff3772712a5e2cfb2a943bfa7ea8500 Jul 24 15:20:02.062849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.062563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:02.062849 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.062773 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:02.062849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.062821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:02.063130 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.062856 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:04.062833525 +0000 UTC m=+5.986758827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:02.063130 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.062897 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:02.063130 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.062947 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:04.062932636 +0000 UTC m=+5.986857928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:02.163635 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.163597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:02.163847 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.163824 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:02.163889 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.163859 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:02.163889 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.163874 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:02.163969 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.163943 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:04.163922395 +0000 UTC m=+6.087847708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:02.476291 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.476142 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:15:00 +0000 UTC" deadline="2028-01-28 11:58:22.320604339 +0000 UTC" Jul 24 15:20:02.476291 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.476206 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13268h38m19.844401818s" Jul 24 15:20:02.530243 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.530201 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:02.530475 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.530335 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:02.530792 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.530768 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:02.530907 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:02.530875 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:02.550991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.550947 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" event={"ID":"ac7f3760-5de1-4c0b-8838-f76677aa4705","Type":"ContainerStarted","Data":"f9501bdfac0de0d004209ec0ddd3f76a0ff3772712a5e2cfb2a943bfa7ea8500"} Jul 24 15:20:02.559411 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.559267 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"bf96f1f5afcfbaaa847b101c3665215e561300731676d5e2fa3ce8b4a8595535"} Jul 24 15:20:02.568109 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.568074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cwk2v" event={"ID":"3313863b-64c5-4ddd-b6d4-bd1401402c52","Type":"ContainerStarted","Data":"c17ba450582643ea9cfa579177b129b0d42bd82d5024ce84aee00c9483771379"} Jul 24 15:20:02.570547 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.570464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerStarted","Data":"16dbb54831747d1f546234c604c8353511d15d1dbe1bfec02c12821226ad292d"} Jul 24 15:20:02.596924 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.596740 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" event={"ID":"cd9f892d-661f-4774-8a30-749cf1235869","Type":"ContainerStarted","Data":"d7e02fcc847f2f3114e52ce70373447333952ce33aa8f05d0ed097facccf7d9e"} Jul 24 15:20:02.609092 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.609053 2564 generic.go:356] "Generic (PLEG): container finished" podID="f69636379af9c8864e9742b149d3b2df" containerID="53bd234f10a4756b32020c5bcec2b5c528a442a70f31e9063e11c062a22cbf48" exitCode=0 Jul 24 15:20:02.609553 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.609355 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" event={"ID":"f69636379af9c8864e9742b149d3b2df","Type":"ContainerDied","Data":"53bd234f10a4756b32020c5bcec2b5c528a442a70f31e9063e11c062a22cbf48"} Jul 24 15:20:02.627335 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.627306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" event={"ID":"414259330d98a4b19eb5aa9f7c16c705","Type":"ContainerStarted","Data":"849187e3defa7e4e61a20a0c7a02c38ebaddbc919ee5d42594807a4928df87b7"} Jul 24 15:20:02.646667 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.643485 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rk7tm" event={"ID":"8aadb56a-0ce2-48d2-b3f1-e874409cd867","Type":"ContainerStarted","Data":"a22066232612763a034ba5c194921450c242bf4ca4e3991ab30faac635ddceb6"} Jul 24 15:20:02.652279 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:02.650044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dhhwh" event={"ID":"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47","Type":"ContainerStarted","Data":"7b3b8d9998b15a6f8a56b503f7b5599eb6c8bac4280e301b335da4ef4b428d73"} Jul 24 15:20:03.529086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:03.528625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:03.529086 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:03.528796 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:03.664258 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:03.664114 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" event={"ID":"f69636379af9c8864e9742b149d3b2df","Type":"ContainerStarted","Data":"9ccfb84abe52ae13d3c8e2165b6b658fd050790d8876b6f804e926e6a6589dc5"} Jul 24 15:20:03.681937 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:03.681014 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-102.ec2.internal" podStartSLOduration=4.680991617 podStartE2EDuration="4.680991617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:02.643461499 +0000 UTC m=+4.567386809" watchObservedRunningTime="2026-07-24 15:20:03.680991617 +0000 UTC m=+5.604916925" Jul 24 15:20:04.079872 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:04.079835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:04.080064 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:04.079889 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:04.080064 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.080022 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:04.080064 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.080024 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:04.080225 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.080089 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:08.080070524 +0000 UTC m=+10.003995813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:04.080225 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.080109 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:08.080099465 +0000 UTC m=+10.004024757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:04.180863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:04.180820 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:04.181033 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.180991 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:04.181033 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.181017 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:04.181033 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.181030 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:04.181497 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.181428 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:08.181400505 +0000 UTC m=+10.105325811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:04.531406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:04.530405 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:04.531406 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.530552 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:04.531406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:04.530673 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:04.531406 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:04.530765 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:05.528462 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:05.528430 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:05.528643 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:05.528554 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:06.532274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:06.532207 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:06.532274 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:06.532230 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:06.532747 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:06.532344 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:06.532747 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:06.532442 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:07.528486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:07.528448 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:07.528709 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:07.528598 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:08.114834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:08.114799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:08.115290 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:08.114844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:08.115290 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.114988 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:08.115290 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.115046 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:16.115027804 +0000 UTC m=+18.038953110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:08.115471 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.115439 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:08.115522 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.115493 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:16.115474668 +0000 UTC m=+18.039399959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:08.215222 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:08.215181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:08.215401 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.215333 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:08.215401 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.215353 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:08.215401 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.215367 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:08.215545 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.215418 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:16.21540259 +0000 UTC m=+18.139327876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:08.529435 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:08.529341 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:08.529435 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:08.529372 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:08.529634 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.529501 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:08.529634 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:08.529602 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:09.528086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:09.528053 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:09.528517 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:09.528183 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:10.530622 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:10.530588 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:10.531076 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:10.530718 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:10.531076 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:10.530878 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:10.531076 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:10.530959 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:11.527638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:11.527605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:11.527813 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:11.527730 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:12.528079 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:12.528041 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:12.528079 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:12.528082 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:12.528555 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:12.528172 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:12.528555 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:12.528296 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:13.528360 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:13.528317 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:13.528825 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:13.528458 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:14.528018 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:14.527986 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:14.528190 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:14.528103 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:14.528470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:14.528454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:14.528896 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:14.528547 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:15.528162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:15.528125 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:15.528372 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:15.528257 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:16.178304 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:16.178267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:16.178765 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:16.178315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:16.178765 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.178441 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:16.178765 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.178450 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:16.178765 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.178517 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:32.17849575 +0000 UTC m=+34.102421039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:16.178765 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.178537 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:32.178527741 +0000 UTC m=+34.102453029 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:16.279698 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:16.279667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:16.279884 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.279864 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:16.279947 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.279889 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:16.279947 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.279902 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:16.280040 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.279969 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:32.279947762 +0000 UTC m=+34.203873049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:16.528607 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:16.528523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:16.528764 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:16.528523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:16.528764 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.528680 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:16.528764 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:16.528715 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:17.528762 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:17.528716 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:17.529205 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:17.528843 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:18.528954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:18.528920 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:18.529384 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:18.528964 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:18.529384 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:18.529018 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:18.529384 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:18.529056 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:19.528116 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:19.528083 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:19.528303 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:19.528217 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:20.528514 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:20.528482 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:20.529139 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:20.528520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:20.529139 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:20.528616 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:20.529139 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:20.528783 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:21.528366 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.527995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:21.528511 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:21.528465 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:21.704036 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.703942 2564 generic.go:356] "Generic (PLEG): container finished" podID="f7ca19f5-babd-47c0-a92b-be5d34696cd6" containerID="a9215f6cf524b56215091870c1e0686254f81a1aba981a10160460f5a0ec823a" exitCode=0 Jul 24 15:20:21.704899 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.704030 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerDied","Data":"a9215f6cf524b56215091870c1e0686254f81a1aba981a10160460f5a0ec823a"} Jul 24 15:20:21.705731 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.705664 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" event={"ID":"cd9f892d-661f-4774-8a30-749cf1235869","Type":"ContainerStarted","Data":"969500954bc7dcf7a6a0a9e2a0fac1e90a7a60ac7182572f20e923b8406403c1"} Jul 24 15:20:21.707513 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.707479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rk7tm" event={"ID":"8aadb56a-0ce2-48d2-b3f1-e874409cd867","Type":"ContainerStarted","Data":"601ffbc43a31144c4030cee73ed8ebfa3d00f764998ffb41910f596fc643ded8"} Jul 24 15:20:21.708969 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.708942 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dhhwh" event={"ID":"d2699c8d-2cd9-44e3-9ada-1885e5bc0a47","Type":"ContainerStarted","Data":"8e9faaf41ddbb044646ed1d8a1f1e067a80644a58e01c8053bdd02799d65d120"} Jul 24 15:20:21.710719 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.710694 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tn7vd" event={"ID":"9d9172f6-f70c-46db-8607-b3413f6afb8d","Type":"ContainerStarted","Data":"4d7a442276bea6c9866e72462b409c44c6431cfaace4d3fdbae60683c65e1427"} Jul 24 15:20:21.712202 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.712177 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" event={"ID":"ac7f3760-5de1-4c0b-8838-f76677aa4705","Type":"ContainerStarted","Data":"822c85180036ed1b05e4b78b28cd5bdcbd24d335ef738276de35a16def5182d3"} Jul 24 15:20:21.715571 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.715547 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"dc5dd8e948a5054fb1de3eab4437ed2c2342d5de0d01689af1133b8a45f4431e"} Jul 24 15:20:21.715692 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.715627 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"99fc370a6d3e7f361efedda95acc02a56adf5db842f5736f90d808cf27417c2c"} Jul 24 15:20:21.715765 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.715705 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"fbc6613fd0904ab7af4ade7e86973a7a9138c562790f627dd6fd8003809707f4"} Jul 24 15:20:21.715765 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.715752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"9a1a79a06eef3f86391895dc80eaffd394d4e406c602e4752bc7e6984eb3368c"} Jul 24 15:20:21.715849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.715790 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"c985842100aa9b5c0ea0c2c511b98a66d51800f214b9d0cdf2d91eee4e516d1e"} Jul 24 15:20:21.715849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.715822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"bbece1e5ab25bf6a77d0088bb5feb8eed5fa2dd33a467598b8b1c239700c39de"} Jul 24 15:20:21.717078 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.717055 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cwk2v" event={"ID":"3313863b-64c5-4ddd-b6d4-bd1401402c52","Type":"ContainerStarted","Data":"054cec862fe27224b842c8b644d4a4c90685f491242388b341803e3e8cc37581"} Jul 24 15:20:21.727119 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.727079 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-102.ec2.internal" podStartSLOduration=22.727065958 podStartE2EDuration="22.727065958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:03.681892985 +0000 UTC m=+5.605818297" watchObservedRunningTime="2026-07-24 15:20:21.727065958 +0000 UTC m=+23.650991288" Jul 24 15:20:21.741767 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.741718 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dhhwh" podStartSLOduration=4.568676033 podStartE2EDuration="23.741704941s" totalImagesPullingTime="19.173028908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:21.741152472 +0000 UTC m=+23.665077796" watchObservedRunningTime="2026-07-24 15:20:21.741704941 +0000 UTC m=+23.665630249" Jul 24 15:20:21.759718 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.759672 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rk7tm" podStartSLOduration=4.525124183 podStartE2EDuration="23.759610107s" totalImagesPullingTime="19.234485924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:21.759558235 +0000 UTC m=+23.683483551" watchObservedRunningTime="2026-07-24 15:20:21.759610107 +0000 UTC m=+23.683535440" Jul 24 15:20:21.777824 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.777775 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9g2rg" podStartSLOduration=4.587409453 podStartE2EDuration="23.777761708s" totalImagesPullingTime="19.190352255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:21.777011344 +0000 UTC m=+23.700936676" watchObservedRunningTime="2026-07-24 15:20:21.777761708 +0000 UTC m=+23.701687020" Jul 24 15:20:21.791931 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.791871 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tn7vd" podStartSLOduration=4.598225288 podStartE2EDuration="23.791850992s" totalImagesPullingTime="19.193625704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:21.791580464 +0000 UTC m=+23.715505793" watchObservedRunningTime="2026-07-24 15:20:21.791850992 +0000 UTC m=+23.715776302" Jul 24 15:20:21.807037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:21.806986 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cwk2v" podStartSLOduration=4.631502956 podStartE2EDuration="23.80696879s" totalImagesPullingTime="19.175465834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:21.806876584 +0000 UTC m=+23.730801890" watchObservedRunningTime="2026-07-24 15:20:21.80696879 +0000 UTC m=+23.730894097" Jul 24 15:20:22.140440 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.140411 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 15:20:22.500264 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.500099 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T15:20:22.140435086Z","UUID":"f3cf50c3-b5b1-4801-88e5-3675efa5becf","Handler":null,"Name":"","Endpoint":""} Jul 24 15:20:22.501942 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.501795 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 15:20:22.501942 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.501830 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 15:20:22.528359 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.528321 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:22.528359 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.528353 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:22.528582 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:22.528444 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:22.528644 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:22.528574 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:22.722009 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.721974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8jk4r" event={"ID":"95805928-870f-43ab-ba4a-7d86e2e7836f","Type":"ContainerStarted","Data":"4040783125e83b833caddedb964e21ebc90eee5e51d905f4efe09821aee32982"} Jul 24 15:20:22.724255 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.724191 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" event={"ID":"ac7f3760-5de1-4c0b-8838-f76677aa4705","Type":"ContainerStarted","Data":"f3103dbe6a66ea11ce250e0aa741887f6390b5c60ef9dae315a0b3c6409d61c5"} Jul 24 15:20:22.737906 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:22.737856 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8jk4r" podStartSLOduration=5.559946947 podStartE2EDuration="24.737840743s" totalImagesPullingTime="19.177893796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:22.737440029 +0000 UTC m=+24.661365366" watchObservedRunningTime="2026-07-24 15:20:22.737840743 +0000 UTC m=+24.661766051" Jul 24 15:20:23.099140 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:23.099093 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:23.099780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:23.099755 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:23.528264 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:23.528237 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:23.528408 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:23.528374 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:23.729163 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:23.729074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" event={"ID":"ac7f3760-5de1-4c0b-8838-f76677aa4705","Type":"ContainerStarted","Data":"61cb641717924df845085ba2457c457f2ecdff3b18c226eae556b0126d7baa98"} Jul 24 15:20:23.732138 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:23.732112 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"c4a5ccfe0e0281acb78a616f63e2f2e1909a1206870b5c3d625d5e6e18952a80"} Jul 24 15:20:23.746987 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:23.746945 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8nldh" podStartSLOduration=4.173336522 podStartE2EDuration="25.746930622s" totalImagesPullingTime="21.5735941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:23.746496426 +0000 UTC m=+25.670421744" watchObservedRunningTime="2026-07-24 15:20:23.746930622 +0000 UTC m=+25.670855930" Jul 24 15:20:24.528013 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:24.527975 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:24.528306 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:24.528102 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:24.528306 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:24.528284 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:24.528459 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:24.528426 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:24.734729 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:24.734690 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 24 15:20:25.528882 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:25.528668 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:25.529071 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:25.528986 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:26.527713 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.527620 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:26.527713 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.527631 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:26.528157 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:26.527774 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:26.528157 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:26.527846 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:26.745357 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.745276 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" event={"ID":"6755c18e-292e-4b86-b685-41bd508fdcfc","Type":"ContainerStarted","Data":"5183f1c140d3c8f6058a958a30413eb5d412e2ed022d9c1e0c277f24e3b64ca0"} Jul 24 15:20:26.745755 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.745738 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:26.745832 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.745815 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:26.764505 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.764191 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:26.769306 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.769236 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:26.773801 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:26.773613 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" podStartSLOduration=9.542065937 podStartE2EDuration="28.77359815s" totalImagesPullingTime="19.231532213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:26.773062028 +0000 UTC m=+28.696987362" watchObservedRunningTime="2026-07-24 15:20:26.77359815 +0000 UTC m=+28.697523458" Jul 24 15:20:27.388405 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.388149 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:27.388585 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.388498 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 24 15:20:27.389242 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.389218 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cwk2v" Jul 24 15:20:27.495936 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.495902 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:20:27.528396 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.528368 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:27.528985 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:27.528471 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:27.748772 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.748690 2564 generic.go:356] "Generic (PLEG): container finished" podID="f7ca19f5-babd-47c0-a92b-be5d34696cd6" containerID="bfb416eb7efc31e5d7c1856c40f75c5cb171c7cccedd3df943919e3872d4bec3" exitCode=0 Jul 24 15:20:27.748906 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:27.748780 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerDied","Data":"bfb416eb7efc31e5d7c1856c40f75c5cb171c7cccedd3df943919e3872d4bec3"} Jul 24 15:20:28.528503 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.528471 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:28.528990 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:28.528584 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:28.528990 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.528626 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:28.528990 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:28.528765 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:28.680479 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.680443 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4sbjk"] Jul 24 15:20:28.683380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.683350 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24625"] Jul 24 15:20:28.684549 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.684526 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9v4nd"] Jul 24 15:20:28.684685 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.684672 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:28.684818 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:28.684796 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:28.753827 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.753801 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:28.753827 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.753802 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerStarted","Data":"5d9974d89dd8e166ea6d51557a066cd53b7100c0afaaac979b0088a2ccb3e477"} Jul 24 15:20:28.754003 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:28.753919 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:28.754003 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:28.753933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:28.754347 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:28.754323 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:29.761091 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:29.761061 2564 generic.go:356] "Generic (PLEG): container finished" podID="f7ca19f5-babd-47c0-a92b-be5d34696cd6" containerID="5d9974d89dd8e166ea6d51557a066cd53b7100c0afaaac979b0088a2ccb3e477" exitCode=0 Jul 24 15:20:29.761442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:29.761108 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerDied","Data":"5d9974d89dd8e166ea6d51557a066cd53b7100c0afaaac979b0088a2ccb3e477"} Jul 24 15:20:30.528465 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:30.528441 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:30.528592 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:30.528470 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:30.528592 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:30.528441 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:30.528592 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:30.528570 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:30.528770 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:30.528687 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:30.528770 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:30.528740 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:30.766229 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:30.766198 2564 generic.go:356] "Generic (PLEG): container finished" podID="f7ca19f5-babd-47c0-a92b-be5d34696cd6" containerID="48455b13ef017511d292739e2f1d9972ad7d2a7e7b8578acea97c34cbe49742b" exitCode=0 Jul 24 15:20:30.766642 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:30.766240 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerDied","Data":"48455b13ef017511d292739e2f1d9972ad7d2a7e7b8578acea97c34cbe49742b"} Jul 24 15:20:32.190256 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:32.190215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:32.190256 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:32.190256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:32.190754 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.190371 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:32.190754 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.190376 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:32.190754 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.190440 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs podName:8213e2c5-8b95-406f-b783-5616c32452a9 nodeName:}" failed. No retries permitted until 2026-07-24 15:21:04.190420742 +0000 UTC m=+66.114346028 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs") pod "network-metrics-daemon-9v4nd" (UID: "8213e2c5-8b95-406f-b783-5616c32452a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:20:32.190754 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.190455 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret podName:4d8711f9-2375-40fb-8a92-ba56c25afc17 nodeName:}" failed. No retries permitted until 2026-07-24 15:21:04.190448229 +0000 UTC m=+66.114373516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret") pod "global-pull-secret-syncer-24625" (UID: "4d8711f9-2375-40fb-8a92-ba56c25afc17") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:20:32.290641 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:32.290604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:32.290821 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.290785 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:20:32.290821 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.290811 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:20:32.290899 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.290827 2564 projected.go:196] Error preparing data for projected volume kube-api-access-gqxtk for pod openshift-network-diagnostics/network-check-target-4sbjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:32.290899 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.290881 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk podName:a979fc68-685b-4c81-9857-6f490f56f885 nodeName:}" failed. No retries permitted until 2026-07-24 15:21:04.290866692 +0000 UTC m=+66.214791979 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqxtk" (UniqueName: "kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk") pod "network-check-target-4sbjk" (UID: "a979fc68-685b-4c81-9857-6f490f56f885") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:20:32.528510 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:32.528037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:32.528510 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:32.528071 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:32.528510 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:32.528037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:32.528510 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.528155 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbjk" podUID="a979fc68-685b-4c81-9857-6f490f56f885" Jul 24 15:20:32.528510 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.528238 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9v4nd" podUID="8213e2c5-8b95-406f-b783-5616c32452a9" Jul 24 15:20:32.528510 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:32.528350 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24625" podUID="4d8711f9-2375-40fb-8a92-ba56c25afc17" Jul 24 15:20:34.444813 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.444782 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-102.ec2.internal" event="NodeReady" Jul 24 15:20:34.445293 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.444945 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 15:20:34.481206 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.481094 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf"] Jul 24 15:20:34.484066 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.484030 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" Jul 24 15:20:34.488343 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.488299 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 15:20:34.488747 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.488725 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 15:20:34.488931 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.488797 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gxt9v\"" Jul 24 15:20:34.489504 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.489483 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7s5hw"] Jul 24 15:20:34.491847 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.491744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.494300 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.494148 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-twsks\"" Jul 24 15:20:34.494300 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.494229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 15:20:34.494631 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.494612 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 15:20:34.499252 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.499215 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kfq8b"] Jul 24 15:20:34.502549 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.502527 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf"] Jul 24 15:20:34.502642 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.502625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.505195 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.505173 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 15:20:34.505304 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.505224 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 15:20:34.505304 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.505264 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 15:20:34.505565 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.505543 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8tm4p\"" Jul 24 15:20:34.507803 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.507780 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7s5hw"] Jul 24 15:20:34.508427 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.508404 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfq8b"] Jul 24 15:20:34.527717 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.527680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:20:34.527878 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.527680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:20:34.527878 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.527690 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:20:34.530616 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.530593 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-75fgn\"" Jul 24 15:20:34.530756 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.530593 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 15:20:34.530756 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.530598 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 15:20:34.530756 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.530719 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hdd4d\"" Jul 24 15:20:34.606335 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606290 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fdxl\" (UniqueName: \"kubernetes.io/projected/c9911024-c893-4476-a99e-a6eb2f02a347-kube-api-access-9fdxl\") pod \"network-check-source-7d88cb55d8-4jlnf\" (UID: \"c9911024-c893-4476-a99e-a6eb2f02a347\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" Jul 24 15:20:34.606335 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606338 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b075ac19-067d-4c2c-bce6-7189606d8fdf-config-volume\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.606542 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86cae9d7-98c7-468d-9d14-7d368de5e8a8-cert\") pod \"ingress-canary-kfq8b\" (UID: \"86cae9d7-98c7-468d-9d14-7d368de5e8a8\") " pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.606542 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66m4t\" (UniqueName: \"kubernetes.io/projected/86cae9d7-98c7-468d-9d14-7d368de5e8a8-kube-api-access-66m4t\") pod \"ingress-canary-kfq8b\" (UID: \"86cae9d7-98c7-468d-9d14-7d368de5e8a8\") " pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.606542 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606503 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.606710 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b075ac19-067d-4c2c-bce6-7189606d8fdf-tmp-dir\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.606710 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.606617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkdpq\" (UniqueName: \"kubernetes.io/projected/b075ac19-067d-4c2c-bce6-7189606d8fdf-kube-api-access-zkdpq\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.707578 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86cae9d7-98c7-468d-9d14-7d368de5e8a8-cert\") pod \"ingress-canary-kfq8b\" (UID: \"86cae9d7-98c7-468d-9d14-7d368de5e8a8\") " pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707595 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66m4t\" (UniqueName: \"kubernetes.io/projected/86cae9d7-98c7-468d-9d14-7d368de5e8a8-kube-api-access-66m4t\") pod \"ingress-canary-kfq8b\" (UID: \"86cae9d7-98c7-468d-9d14-7d368de5e8a8\") " pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707625 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b075ac19-067d-4c2c-bce6-7189606d8fdf-tmp-dir\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkdpq\" (UniqueName: \"kubernetes.io/projected/b075ac19-067d-4c2c-bce6-7189606d8fdf-kube-api-access-zkdpq\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fdxl\" (UniqueName: \"kubernetes.io/projected/c9911024-c893-4476-a99e-a6eb2f02a347-kube-api-access-9fdxl\") pod \"network-check-source-7d88cb55d8-4jlnf\" (UID: \"c9911024-c893-4476-a99e-a6eb2f02a347\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.707776 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b075ac19-067d-4c2c-bce6-7189606d8fdf-config-volume\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.707863 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:34.707793 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 15:20:34.708204 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:34.707876 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls podName:b075ac19-067d-4c2c-bce6-7189606d8fdf nodeName:}" failed. No retries permitted until 2026-07-24 15:20:35.207854889 +0000 UTC m=+37.131780196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls") pod "dns-default-7s5hw" (UID: "b075ac19-067d-4c2c-bce6-7189606d8fdf") : secret "dns-default-metrics-tls" not found Jul 24 15:20:34.708204 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.708041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b075ac19-067d-4c2c-bce6-7189606d8fdf-tmp-dir\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.708520 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.708500 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b075ac19-067d-4c2c-bce6-7189606d8fdf-config-volume\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.712873 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.712852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86cae9d7-98c7-468d-9d14-7d368de5e8a8-cert\") pod \"ingress-canary-kfq8b\" (UID: \"86cae9d7-98c7-468d-9d14-7d368de5e8a8\") " pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.717577 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.717552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkdpq\" (UniqueName: \"kubernetes.io/projected/b075ac19-067d-4c2c-bce6-7189606d8fdf-kube-api-access-zkdpq\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:34.717719 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.717623 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66m4t\" (UniqueName: \"kubernetes.io/projected/86cae9d7-98c7-468d-9d14-7d368de5e8a8-kube-api-access-66m4t\") pod \"ingress-canary-kfq8b\" (UID: \"86cae9d7-98c7-468d-9d14-7d368de5e8a8\") " pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:34.718480 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.718461 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fdxl\" (UniqueName: \"kubernetes.io/projected/c9911024-c893-4476-a99e-a6eb2f02a347-kube-api-access-9fdxl\") pod \"network-check-source-7d88cb55d8-4jlnf\" (UID: \"c9911024-c893-4476-a99e-a6eb2f02a347\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" Jul 24 15:20:34.796570 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.796484 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" Jul 24 15:20:34.815056 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:34.815018 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfq8b" Jul 24 15:20:35.212281 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.212009 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:35.212446 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:35.212188 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 15:20:35.212501 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:35.212458 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls podName:b075ac19-067d-4c2c-bce6-7189606d8fdf nodeName:}" failed. No retries permitted until 2026-07-24 15:20:36.21243352 +0000 UTC m=+38.136358828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls") pod "dns-default-7s5hw" (UID: "b075ac19-067d-4c2c-bce6-7189606d8fdf") : secret "dns-default-metrics-tls" not found Jul 24 15:20:35.275509 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.275471 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfq8b"] Jul 24 15:20:35.278151 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:35.278099 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cae9d7_98c7_468d_9d14_7d368de5e8a8.slice/crio-f558df7050de1e92d94a06cb03c69f5c611fdb19b57100e027f7a7b54ec65156 WatchSource:0}: Error finding container f558df7050de1e92d94a06cb03c69f5c611fdb19b57100e027f7a7b54ec65156: Status 404 returned error can't find the container with id f558df7050de1e92d94a06cb03c69f5c611fdb19b57100e027f7a7b54ec65156 Jul 24 15:20:35.279590 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.279567 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf"] Jul 24 15:20:35.292529 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:35.292507 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9911024_c893_4476_a99e_a6eb2f02a347.slice/crio-4b2fcb947e78c367243bb902a30e35e59608d457df4adce283bda687fe1bfffa WatchSource:0}: Error finding container 4b2fcb947e78c367243bb902a30e35e59608d457df4adce283bda687fe1bfffa: Status 404 returned error can't find the container with id 4b2fcb947e78c367243bb902a30e35e59608d457df4adce283bda687fe1bfffa Jul 24 15:20:35.781978 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.781941 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfq8b" event={"ID":"86cae9d7-98c7-468d-9d14-7d368de5e8a8","Type":"ContainerStarted","Data":"f558df7050de1e92d94a06cb03c69f5c611fdb19b57100e027f7a7b54ec65156"} Jul 24 15:20:35.783239 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.783215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" event={"ID":"c9911024-c893-4476-a99e-a6eb2f02a347","Type":"ContainerStarted","Data":"4b2fcb947e78c367243bb902a30e35e59608d457df4adce283bda687fe1bfffa"} Jul 24 15:20:35.786360 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.786336 2564 generic.go:356] "Generic (PLEG): container finished" podID="f7ca19f5-babd-47c0-a92b-be5d34696cd6" containerID="e868ebf3a39a2e259cd332aada7fc2efc89932c77b51ec939d08d70724cd632e" exitCode=0 Jul 24 15:20:35.786478 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:35.786382 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerDied","Data":"e868ebf3a39a2e259cd332aada7fc2efc89932c77b51ec939d08d70724cd632e"} Jul 24 15:20:36.220288 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:36.220255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:36.220495 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:36.220444 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 15:20:36.220644 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:36.220628 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls podName:b075ac19-067d-4c2c-bce6-7189606d8fdf nodeName:}" failed. No retries permitted until 2026-07-24 15:20:38.2205233 +0000 UTC m=+40.144448588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls") pod "dns-default-7s5hw" (UID: "b075ac19-067d-4c2c-bce6-7189606d8fdf") : secret "dns-default-metrics-tls" not found Jul 24 15:20:36.792874 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:36.792835 2564 generic.go:356] "Generic (PLEG): container finished" podID="f7ca19f5-babd-47c0-a92b-be5d34696cd6" containerID="5d7241f9a96024d861f0efcc3f0a0a8ed591e3baf29d754f81b5bf134b65faef" exitCode=0 Jul 24 15:20:36.793302 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:36.792887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerDied","Data":"5d7241f9a96024d861f0efcc3f0a0a8ed591e3baf29d754f81b5bf134b65faef"} Jul 24 15:20:37.800011 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:37.799753 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" event={"ID":"f7ca19f5-babd-47c0-a92b-be5d34696cd6","Type":"ContainerStarted","Data":"6ce4c0c4337021ceb9d821ac9a92aab6ceee63829106d5e92b8a7ecd038046cf"} Jul 24 15:20:37.834606 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:37.834543 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9fjwh" podStartSLOduration=9.681423588 podStartE2EDuration="39.834520572s" totalImagesPullingTime="30.153096984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:37.833045503 +0000 UTC m=+39.756970853" watchObservedRunningTime="2026-07-24 15:20:37.834520572 +0000 UTC m=+39.758445884" Jul 24 15:20:38.239133 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.239090 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:38.239319 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:38.239261 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 15:20:38.239381 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:38.239336 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls podName:b075ac19-067d-4c2c-bce6-7189606d8fdf nodeName:}" failed. No retries permitted until 2026-07-24 15:20:42.239317413 +0000 UTC m=+44.163242700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls") pod "dns-default-7s5hw" (UID: "b075ac19-067d-4c2c-bce6-7189606d8fdf") : secret "dns-default-metrics-tls" not found Jul 24 15:20:38.700790 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.700762 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ftppt"] Jul 24 15:20:38.706184 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.706151 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.709137 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.709104 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 15:20:38.709419 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.709402 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ns8gt\"" Jul 24 15:20:38.710169 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.710150 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 15:20:38.710434 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.710414 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 15:20:38.710535 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.710470 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 15:20:38.722208 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.722175 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftppt"] Jul 24 15:20:38.844565 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.844530 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/092e3a8a-1471-4f54-ab2d-9653c2638d65-crio-socket\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.845157 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.844585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khh6t\" (UniqueName: \"kubernetes.io/projected/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-api-access-khh6t\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.845157 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.844698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.845157 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.844821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/092e3a8a-1471-4f54-ab2d-9653c2638d65-data-volume\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.845157 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.844865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.946123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/092e3a8a-1471-4f54-ab2d-9653c2638d65-crio-socket\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.946186 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khh6t\" (UniqueName: \"kubernetes.io/projected/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-api-access-khh6t\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.946256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:38.946368 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.946400 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/092e3a8a-1471-4f54-ab2d-9653c2638d65-crio-socket\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:38.946428 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:39.446409483 +0000 UTC m=+41.370334788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : secret "insights-runtime-extractor-tls" not found Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.946491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/092e3a8a-1471-4f54-ab2d-9653c2638d65-data-volume\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.946605 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.946515 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.947129 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:38.947109 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 15:20:38.947197 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:38.947187 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:39.447159767 +0000 UTC m=+41.371085065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : configmap "kube-rbac-proxy" not found Jul 24 15:20:38.947251 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.947212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/092e3a8a-1471-4f54-ab2d-9653c2638d65-data-volume\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.964113 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.964043 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khh6t\" (UniqueName: \"kubernetes.io/projected/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-api-access-khh6t\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:38.994142 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:38.994109 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6"] Jul 24 15:20:39.013974 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.013937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6"] Jul 24 15:20:39.014159 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.014100 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" Jul 24 15:20:39.017076 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.017050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-79v5z\"" Jul 24 15:20:39.017076 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.017075 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 15:20:39.017412 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.017392 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 15:20:39.148073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.148035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcdr4\" (UniqueName: \"kubernetes.io/projected/a4ccc847-ece6-440c-bc4b-329f789260c9-kube-api-access-gcdr4\") pod \"migrator-6865c9967-xvsq6\" (UID: \"a4ccc847-ece6-440c-bc4b-329f789260c9\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" Jul 24 15:20:39.249508 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.249424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcdr4\" (UniqueName: \"kubernetes.io/projected/a4ccc847-ece6-440c-bc4b-329f789260c9-kube-api-access-gcdr4\") pod \"migrator-6865c9967-xvsq6\" (UID: \"a4ccc847-ece6-440c-bc4b-329f789260c9\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" Jul 24 15:20:39.258138 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.258111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcdr4\" (UniqueName: \"kubernetes.io/projected/a4ccc847-ece6-440c-bc4b-329f789260c9-kube-api-access-gcdr4\") pod \"migrator-6865c9967-xvsq6\" (UID: \"a4ccc847-ece6-440c-bc4b-329f789260c9\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" Jul 24 15:20:39.326595 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.326561 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" Jul 24 15:20:39.451704 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.451665 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:39.451866 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.451737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:39.451866 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:39.451833 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 15:20:39.451866 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:39.451833 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 15:20:39.451964 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:39.451886 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:40.451872009 +0000 UTC m=+42.375797296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : configmap "kube-rbac-proxy" not found Jul 24 15:20:39.451964 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:39.451898 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:40.451892425 +0000 UTC m=+42.375817711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : secret "insights-runtime-extractor-tls" not found Jul 24 15:20:39.613849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.613815 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6"] Jul 24 15:20:39.617936 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:39.617909 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ccc847_ece6_440c_bc4b_329f789260c9.slice/crio-e0c3760dd67db5da8365a54f24579a921b8514cccbf72811d3ff87fc1eab9bab WatchSource:0}: Error finding container e0c3760dd67db5da8365a54f24579a921b8514cccbf72811d3ff87fc1eab9bab: Status 404 returned error can't find the container with id e0c3760dd67db5da8365a54f24579a921b8514cccbf72811d3ff87fc1eab9bab Jul 24 15:20:39.807363 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.807263 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfq8b" event={"ID":"86cae9d7-98c7-468d-9d14-7d368de5e8a8","Type":"ContainerStarted","Data":"dbfce289d5ca021959a733ab876f2799e3e8e2d33a3c1b68a1a69cb2333c7af8"} Jul 24 15:20:39.808801 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.808766 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" event={"ID":"c9911024-c893-4476-a99e-a6eb2f02a347","Type":"ContainerStarted","Data":"9f4cdb40e0d570b625b90b0dcc2f1b24b0dfe51bf415214467f964e7a3d09702"} Jul 24 15:20:39.809915 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.809896 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" event={"ID":"a4ccc847-ece6-440c-bc4b-329f789260c9","Type":"ContainerStarted","Data":"e0c3760dd67db5da8365a54f24579a921b8514cccbf72811d3ff87fc1eab9bab"} Jul 24 15:20:39.826038 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.825987 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kfq8b" podStartSLOduration=1.622964009 podStartE2EDuration="5.825972595s" totalImagesPullingTime="4.203008586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:39.825124516 +0000 UTC m=+41.749049824" watchObservedRunningTime="2026-07-24 15:20:39.825972595 +0000 UTC m=+41.749897954" Jul 24 15:20:39.839914 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:39.839874 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4jlnf" podStartSLOduration=1.642338104 podStartE2EDuration="5.839860078s" totalImagesPullingTime="4.197521974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:39.839486373 +0000 UTC m=+41.763411685" watchObservedRunningTime="2026-07-24 15:20:39.839860078 +0000 UTC m=+41.763785364" Jul 24 15:20:40.460470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:40.460426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:40.460961 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:40.460516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:40.460961 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:40.460667 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 15:20:40.460961 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:40.460731 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:42.460710176 +0000 UTC m=+44.384635466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : configmap "kube-rbac-proxy" not found Jul 24 15:20:40.461184 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:40.461141 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 15:20:40.461290 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:40.461227 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:42.461208547 +0000 UTC m=+44.385133836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : secret "insights-runtime-extractor-tls" not found Jul 24 15:20:40.898675 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:40.898570 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tn7vd_9d9172f6-f70c-46db-8607-b3413f6afb8d/dns-node-resolver/0.log" Jul 24 15:20:41.184191 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.184126 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-8kskq"] Jul 24 15:20:41.201354 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.201326 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-8kskq"] Jul 24 15:20:41.201487 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.201373 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.204023 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.203980 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 15:20:41.204023 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.204022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p6dg8\"" Jul 24 15:20:41.204204 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.204034 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 15:20:41.205345 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.205324 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 15:20:41.205345 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.205332 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 15:20:41.369399 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.369376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/640625ec-3896-4dea-a316-f961f2147ab7-signing-key\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.369518 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.369438 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6hrg\" (UniqueName: \"kubernetes.io/projected/640625ec-3896-4dea-a316-f961f2147ab7-kube-api-access-h6hrg\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.369518 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.369499 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/640625ec-3896-4dea-a316-f961f2147ab7-signing-cabundle\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.470565 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.470530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/640625ec-3896-4dea-a316-f961f2147ab7-signing-cabundle\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.471012 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.470604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/640625ec-3896-4dea-a316-f961f2147ab7-signing-key\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.471012 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.470681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6hrg\" (UniqueName: \"kubernetes.io/projected/640625ec-3896-4dea-a316-f961f2147ab7-kube-api-access-h6hrg\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.471380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.471355 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/640625ec-3896-4dea-a316-f961f2147ab7-signing-cabundle\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.473474 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.473439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/640625ec-3896-4dea-a316-f961f2147ab7-signing-key\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.480809 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.480790 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6hrg\" (UniqueName: \"kubernetes.io/projected/640625ec-3896-4dea-a316-f961f2147ab7-kube-api-access-h6hrg\") pod \"service-ca-658d7b7c78-8kskq\" (UID: \"640625ec-3896-4dea-a316-f961f2147ab7\") " pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.498268 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.498246 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dhhwh_d2699c8d-2cd9-44e3-9ada-1885e5bc0a47/node-ca/0.log" Jul 24 15:20:41.510525 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.510510 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" Jul 24 15:20:41.655189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.655160 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-8kskq"] Jul 24 15:20:41.658226 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:41.658198 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640625ec_3896_4dea_a316_f961f2147ab7.slice/crio-22f22a11b9ed7b918ed4c298afd34c3fd986810b50914a5675dbb44f09ec7317 WatchSource:0}: Error finding container 22f22a11b9ed7b918ed4c298afd34c3fd986810b50914a5675dbb44f09ec7317: Status 404 returned error can't find the container with id 22f22a11b9ed7b918ed4c298afd34c3fd986810b50914a5675dbb44f09ec7317 Jul 24 15:20:41.817126 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.817014 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" event={"ID":"a4ccc847-ece6-440c-bc4b-329f789260c9","Type":"ContainerStarted","Data":"3d1fcf11a8bbf19480560a7df678c5de4a4984a26edab5d57cc399f0daa6486d"} Jul 24 15:20:41.817126 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.817091 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" event={"ID":"a4ccc847-ece6-440c-bc4b-329f789260c9","Type":"ContainerStarted","Data":"f7e5f93e72dfb3a88e406dbd32bf0e615ab023449983a7dd329d2ca3769d1274"} Jul 24 15:20:41.818170 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:41.818146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" event={"ID":"640625ec-3896-4dea-a316-f961f2147ab7","Type":"ContainerStarted","Data":"22f22a11b9ed7b918ed4c298afd34c3fd986810b50914a5675dbb44f09ec7317"} Jul 24 15:20:42.277709 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:42.277667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:42.277903 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:42.277810 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 15:20:42.277903 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:42.277874 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls podName:b075ac19-067d-4c2c-bce6-7189606d8fdf nodeName:}" failed. No retries permitted until 2026-07-24 15:20:50.277858307 +0000 UTC m=+52.201783607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls") pod "dns-default-7s5hw" (UID: "b075ac19-067d-4c2c-bce6-7189606d8fdf") : secret "dns-default-metrics-tls" not found Jul 24 15:20:42.479570 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:42.479528 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:42.480000 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:42.479637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:42.480000 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:42.479727 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 15:20:42.480000 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:42.479806 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:46.479783944 +0000 UTC m=+48.403709265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : secret "insights-runtime-extractor-tls" not found Jul 24 15:20:42.480000 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:42.479727 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 15:20:42.480000 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:42.479863 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:46.479848204 +0000 UTC m=+48.403773490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : configmap "kube-rbac-proxy" not found Jul 24 15:20:42.496575 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:42.496525 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfq8b_86cae9d7-98c7-468d-9d14-7d368de5e8a8/serve-healthcheck-canary/0.log" Jul 24 15:20:44.826791 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:44.826757 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" event={"ID":"640625ec-3896-4dea-a316-f961f2147ab7","Type":"ContainerStarted","Data":"818e933feb6e92d3ea585e4955e6b8b452718febeee8df9b42389722660a84a9"} Jul 24 15:20:44.844261 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:44.844216 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xvsq6" podStartSLOduration=5.1284689199999995 podStartE2EDuration="6.844200066s" totalImagesPullingTime="1.715731146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:41.845701421 +0000 UTC m=+43.769626730" watchObservedRunningTime="2026-07-24 15:20:44.844200066 +0000 UTC m=+46.768125374" Jul 24 15:20:44.844389 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:44.844291 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-8kskq" podStartSLOduration=1.40151951 podStartE2EDuration="3.844286609s" totalImagesPullingTime="2.442767099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:44.84331512 +0000 UTC m=+46.767240424" watchObservedRunningTime="2026-07-24 15:20:44.844286609 +0000 UTC m=+46.768211916" Jul 24 15:20:46.511402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:46.511358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:46.511939 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:46.511421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:46.511939 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:46.511506 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 15:20:46.511939 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:46.511520 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 15:20:46.511939 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:46.511562 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:54.511547714 +0000 UTC m=+56.435473003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : configmap "kube-rbac-proxy" not found Jul 24 15:20:46.511939 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:46.511584 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:20:54.511568418 +0000 UTC m=+56.435493704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : secret "insights-runtime-extractor-tls" not found Jul 24 15:20:50.343481 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:50.343444 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:50.345938 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:50.345918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b075ac19-067d-4c2c-bce6-7189606d8fdf-metrics-tls\") pod \"dns-default-7s5hw\" (UID: \"b075ac19-067d-4c2c-bce6-7189606d8fdf\") " pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:50.404892 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:50.404858 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:50.536520 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:50.536492 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7s5hw"] Jul 24 15:20:50.540159 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:20:50.540116 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb075ac19_067d_4c2c_bce6_7189606d8fdf.slice/crio-ae9951feb4a59be141c11aa8ea1191c5caf8f3dea67fbc0b41525c8501e005db WatchSource:0}: Error finding container ae9951feb4a59be141c11aa8ea1191c5caf8f3dea67fbc0b41525c8501e005db: Status 404 returned error can't find the container with id ae9951feb4a59be141c11aa8ea1191c5caf8f3dea67fbc0b41525c8501e005db Jul 24 15:20:50.844039 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:50.843987 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7s5hw" event={"ID":"b075ac19-067d-4c2c-bce6-7189606d8fdf","Type":"ContainerStarted","Data":"ae9951feb4a59be141c11aa8ea1191c5caf8f3dea67fbc0b41525c8501e005db"} Jul 24 15:20:52.851529 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:52.851444 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7s5hw" event={"ID":"b075ac19-067d-4c2c-bce6-7189606d8fdf","Type":"ContainerStarted","Data":"577e03afe5492f53c95a332281ee5cd7effaee4606507a0f6303ee67310db009"} Jul 24 15:20:52.851529 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:52.851497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7s5hw" event={"ID":"b075ac19-067d-4c2c-bce6-7189606d8fdf","Type":"ContainerStarted","Data":"86b38362a0781970115f6033e4d5900409086d6bb94320475a1c90a46e486b9b"} Jul 24 15:20:52.851961 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:52.851600 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7s5hw" Jul 24 15:20:52.870435 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:52.870389 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7s5hw" podStartSLOduration=16.93770561 podStartE2EDuration="18.87037734s" totalImagesPullingTime="1.93267173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:20:52.870157397 +0000 UTC m=+54.794082719" watchObservedRunningTime="2026-07-24 15:20:52.87037734 +0000 UTC m=+54.794302648" Jul 24 15:20:54.575888 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:54.575854 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:54.576282 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:54.575914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:54.576282 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:54.576018 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 15:20:54.576282 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:20:54.576083 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls podName:092e3a8a-1471-4f54-ab2d-9653c2638d65 nodeName:}" failed. No retries permitted until 2026-07-24 15:21:10.576067318 +0000 UTC m=+72.499992604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftppt" (UID: "092e3a8a-1471-4f54-ab2d-9653c2638d65") : secret "insights-runtime-extractor-tls" not found Jul 24 15:20:54.576414 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:54.576397 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/092e3a8a-1471-4f54-ab2d-9653c2638d65-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:20:58.769763 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:20:58.769735 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ztqv" Jul 24 15:21:01.779812 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.779773 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-b87wr"] Jul 24 15:21:01.782871 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.782848 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z"] Jul 24 15:21:01.782991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.782963 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.785078 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.785060 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.785768 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.785749 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 24 15:21:01.786756 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.786737 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bnnnr\"" Jul 24 15:21:01.786837 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.786772 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 24 15:21:01.788489 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.788468 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 15:21:01.788837 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.788805 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 15:21:01.788928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.788883 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 15:21:01.788928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.788887 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 15:21:01.788928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.788902 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 15:21:01.789273 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.789260 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 15:21:01.789920 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.789885 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 15:21:01.793271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.793251 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-b87wr"] Jul 24 15:21:01.800725 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.800706 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z"] Jul 24 15:21:01.822324 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/07aeeb86-6f77-4012-9f0f-4c6c16c170a2-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-b87wr\" (UID: \"07aeeb86-6f77-4012-9f0f-4c6c16c170a2\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.822425 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/775cb3e7-8368-4096-a294-b2fb0100eac3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.822425 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-ca\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.822425 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqtn4\" (UniqueName: \"kubernetes.io/projected/775cb3e7-8368-4096-a294-b2fb0100eac3-kube-api-access-sqtn4\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.822550 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822423 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.822550 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07aeeb86-6f77-4012-9f0f-4c6c16c170a2-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-b87wr\" (UID: \"07aeeb86-6f77-4012-9f0f-4c6c16c170a2\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.822550 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822502 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.822550 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.822538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-hub\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.922857 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.922823 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07aeeb86-6f77-4012-9f0f-4c6c16c170a2-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-b87wr\" (UID: \"07aeeb86-6f77-4012-9f0f-4c6c16c170a2\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.922857 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.922857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.923092 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.922888 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-hub\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.923092 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.922929 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/07aeeb86-6f77-4012-9f0f-4c6c16c170a2-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-b87wr\" (UID: \"07aeeb86-6f77-4012-9f0f-4c6c16c170a2\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.923092 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.922947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/775cb3e7-8368-4096-a294-b2fb0100eac3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.923092 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.922981 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-ca\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.923092 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.923010 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqtn4\" (UniqueName: \"kubernetes.io/projected/775cb3e7-8368-4096-a294-b2fb0100eac3-kube-api-access-sqtn4\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.923345 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.923139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.923726 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.923701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07aeeb86-6f77-4012-9f0f-4c6c16c170a2-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-b87wr\" (UID: \"07aeeb86-6f77-4012-9f0f-4c6c16c170a2\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.924095 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.924069 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/775cb3e7-8368-4096-a294-b2fb0100eac3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.925769 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.925746 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/07aeeb86-6f77-4012-9f0f-4c6c16c170a2-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-b87wr\" (UID: \"07aeeb86-6f77-4012-9f0f-4c6c16c170a2\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:01.926266 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.926244 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.926328 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.926265 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-hub\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.926328 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.926279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-ca\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.926395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.926354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/775cb3e7-8368-4096-a294-b2fb0100eac3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:01.935306 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:01.935281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqtn4\" (UniqueName: \"kubernetes.io/projected/775cb3e7-8368-4096-a294-b2fb0100eac3-kube-api-access-sqtn4\") pod \"cluster-proxy-proxy-agent-67d4dc7f77-jdk7z\" (UID: \"775cb3e7-8368-4096-a294-b2fb0100eac3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:02.093429 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.093351 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" Jul 24 15:21:02.113466 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.113432 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" Jul 24 15:21:02.229517 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.229395 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-b87wr"] Jul 24 15:21:02.232052 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:02.232026 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07aeeb86_6f77_4012_9f0f_4c6c16c170a2.slice/crio-d362032a5a8b6fa4bbc334106e5f75b8dd9ee7233d8f1b2112aaad12846f3b7a WatchSource:0}: Error finding container d362032a5a8b6fa4bbc334106e5f75b8dd9ee7233d8f1b2112aaad12846f3b7a: Status 404 returned error can't find the container with id d362032a5a8b6fa4bbc334106e5f75b8dd9ee7233d8f1b2112aaad12846f3b7a Jul 24 15:21:02.249352 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.249331 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z"] Jul 24 15:21:02.252028 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:02.252003 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod775cb3e7_8368_4096_a294_b2fb0100eac3.slice/crio-1b9f276872edf591654f9edd564efd1b01629b6976934c38b57a2a61af89b6c3 WatchSource:0}: Error finding container 1b9f276872edf591654f9edd564efd1b01629b6976934c38b57a2a61af89b6c3: Status 404 returned error can't find the container with id 1b9f276872edf591654f9edd564efd1b01629b6976934c38b57a2a61af89b6c3 Jul 24 15:21:02.856021 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.855994 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7s5hw" Jul 24 15:21:02.882447 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.882415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" event={"ID":"07aeeb86-6f77-4012-9f0f-4c6c16c170a2","Type":"ContainerStarted","Data":"d362032a5a8b6fa4bbc334106e5f75b8dd9ee7233d8f1b2112aaad12846f3b7a"} Jul 24 15:21:02.883636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:02.883591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" event={"ID":"775cb3e7-8368-4096-a294-b2fb0100eac3","Type":"ContainerStarted","Data":"1b9f276872edf591654f9edd564efd1b01629b6976934c38b57a2a61af89b6c3"} Jul 24 15:21:03.889572 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:03.889483 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" event={"ID":"07aeeb86-6f77-4012-9f0f-4c6c16c170a2","Type":"ContainerStarted","Data":"cc86755ddc39c9ff8d18252355cbc5d9a1e3373e2c260bc188152ed4544a9d54"} Jul 24 15:21:04.242754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.242629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:21:04.242938 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.242772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:21:04.245488 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.245466 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 15:21:04.245616 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.245509 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 15:21:04.256129 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.256064 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8213e2c5-8b95-406f-b783-5616c32452a9-metrics-certs\") pod \"network-metrics-daemon-9v4nd\" (UID: \"8213e2c5-8b95-406f-b783-5616c32452a9\") " pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:21:04.256914 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.256864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d8711f9-2375-40fb-8a92-ba56c25afc17-original-pull-secret\") pod \"global-pull-secret-syncer-24625\" (UID: \"4d8711f9-2375-40fb-8a92-ba56c25afc17\") " pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:21:04.343285 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.343246 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:21:04.346288 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.346259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqxtk\" (UniqueName: \"kubernetes.io/projected/a979fc68-685b-4c81-9857-6f490f56f885-kube-api-access-gqxtk\") pod \"network-check-target-4sbjk\" (UID: \"a979fc68-685b-4c81-9857-6f490f56f885\") " pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:21:04.543057 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.542976 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hdd4d\"" Jul 24 15:21:04.548405 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.548213 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24625" Jul 24 15:21:04.551357 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.551332 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:21:04.557523 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.557499 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-75fgn\"" Jul 24 15:21:04.564946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.564889 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9v4nd" Jul 24 15:21:04.947276 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.947225 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-b87wr" podStartSLOduration=2.581052035 podStartE2EDuration="3.947202238s" totalImagesPullingTime="1.366150203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:03.913078356 +0000 UTC m=+65.837003662" watchObservedRunningTime="2026-07-24 15:21:04.947202238 +0000 UTC m=+66.871127549" Jul 24 15:21:04.947736 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:04.947705 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9v4nd"] Jul 24 15:21:04.951840 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:04.951803 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8213e2c5_8b95_406f_b783_5616c32452a9.slice/crio-290b67e9f22e17897d218e6cc8f9fd543ba8ab349e08b4cd257d3256e24bcbf2 WatchSource:0}: Error finding container 290b67e9f22e17897d218e6cc8f9fd543ba8ab349e08b4cd257d3256e24bcbf2: Status 404 returned error can't find the container with id 290b67e9f22e17897d218e6cc8f9fd543ba8ab349e08b4cd257d3256e24bcbf2 Jul 24 15:21:05.164949 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:05.164872 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24625"] Jul 24 15:21:05.167970 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:05.167941 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4sbjk"] Jul 24 15:21:05.899294 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:05.899248 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9v4nd" event={"ID":"8213e2c5-8b95-406f-b783-5616c32452a9","Type":"ContainerStarted","Data":"290b67e9f22e17897d218e6cc8f9fd543ba8ab349e08b4cd257d3256e24bcbf2"} Jul 24 15:21:06.221548 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:06.221511 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8711f9_2375_40fb_8a92_ba56c25afc17.slice/crio-85cf3b506ad5798bb93cf1a98e0fa18f2b0e806f407218d5cc43b0f7e229beca WatchSource:0}: Error finding container 85cf3b506ad5798bb93cf1a98e0fa18f2b0e806f407218d5cc43b0f7e229beca: Status 404 returned error can't find the container with id 85cf3b506ad5798bb93cf1a98e0fa18f2b0e806f407218d5cc43b0f7e229beca Jul 24 15:21:06.222340 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:06.222245 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda979fc68_685b_4c81_9857_6f490f56f885.slice/crio-77af9d6181524529a123105949268c1a6b56b87f661e222e1bb38f99fbd3a540 WatchSource:0}: Error finding container 77af9d6181524529a123105949268c1a6b56b87f661e222e1bb38f99fbd3a540: Status 404 returned error can't find the container with id 77af9d6181524529a123105949268c1a6b56b87f661e222e1bb38f99fbd3a540 Jul 24 15:21:06.905221 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.905098 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4sbjk" event={"ID":"a979fc68-685b-4c81-9857-6f490f56f885","Type":"ContainerStarted","Data":"907e2f68e2a808bf2d26f109a9caa5c1af01ab97aa96b48b1103eacc3db55720"} Jul 24 15:21:06.905221 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.905177 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4sbjk" event={"ID":"a979fc68-685b-4c81-9857-6f490f56f885","Type":"ContainerStarted","Data":"77af9d6181524529a123105949268c1a6b56b87f661e222e1bb38f99fbd3a540"} Jul 24 15:21:06.905487 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.905274 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:21:06.906491 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.906461 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24625" event={"ID":"4d8711f9-2375-40fb-8a92-ba56c25afc17","Type":"ContainerStarted","Data":"85cf3b506ad5798bb93cf1a98e0fa18f2b0e806f407218d5cc43b0f7e229beca"} Jul 24 15:21:06.910048 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.910025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" event={"ID":"775cb3e7-8368-4096-a294-b2fb0100eac3","Type":"ContainerStarted","Data":"42f80468ad71cfc1ba69bf72cc0adcde47c4aba0ee6d4d8347e195e12eac1427"} Jul 24 15:21:06.910166 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.910058 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" event={"ID":"775cb3e7-8368-4096-a294-b2fb0100eac3","Type":"ContainerStarted","Data":"de71d7890cdb819b87f7c708c139e7b2ed9e57c130ea3cbb54e02ed87d2859a5"} Jul 24 15:21:06.910166 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.910069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" event={"ID":"775cb3e7-8368-4096-a294-b2fb0100eac3","Type":"ContainerStarted","Data":"a97be741df4949012cd26f4f860d723e68dde5de982e352b4edfe852350d7179"} Jul 24 15:21:06.922295 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:06.922231 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4sbjk" podStartSLOduration=68.922214179 podStartE2EDuration="1m8.922214179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:06.921055956 +0000 UTC m=+68.844981271" watchObservedRunningTime="2026-07-24 15:21:06.922214179 +0000 UTC m=+68.846139499" Jul 24 15:21:07.916391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:07.916353 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9v4nd" event={"ID":"8213e2c5-8b95-406f-b783-5616c32452a9","Type":"ContainerStarted","Data":"abcf4ca4c3f64d179fc68f823a4a0fb73c394ba5a4528e3f9f2696c18a4ac635"} Jul 24 15:21:07.916840 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:07.916450 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9v4nd" event={"ID":"8213e2c5-8b95-406f-b783-5616c32452a9","Type":"ContainerStarted","Data":"2005021c03d282a7bf1aacd5721f0e8cfaad9c965357d888a7b8d5cec61e9c14"} Jul 24 15:21:07.936280 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:07.936231 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d4dc7f77-jdk7z" podStartSLOduration=2.81600274 podStartE2EDuration="6.936214947s" totalImagesPullingTime="4.120212207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:06.942028238 +0000 UTC m=+68.865953549" watchObservedRunningTime="2026-07-24 15:21:07.936214947 +0000 UTC m=+69.860140254" Jul 24 15:21:07.936896 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:07.936867 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9v4nd" podStartSLOduration=67.888938743 podStartE2EDuration="1m9.936854145s" totalImagesPullingTime="2.047915402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:07.935929852 +0000 UTC m=+69.859855187" watchObservedRunningTime="2026-07-24 15:21:07.936854145 +0000 UTC m=+69.860779454" Jul 24 15:21:10.591697 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.591578 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:21:10.594009 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.593986 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/092e3a8a-1471-4f54-ab2d-9653c2638d65-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftppt\" (UID: \"092e3a8a-1471-4f54-ab2d-9653c2638d65\") " pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:21:10.820530 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.820497 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ns8gt\"" Jul 24 15:21:10.828593 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.828576 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftppt" Jul 24 15:21:10.930695 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.930666 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24625" event={"ID":"4d8711f9-2375-40fb-8a92-ba56c25afc17","Type":"ContainerStarted","Data":"ecc07db36af86461de565ada0d588c2a0f33b7d707c286121e477116c2106a99"} Jul 24 15:21:10.947258 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.947204 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-24625" podStartSLOduration=68.92556127 podStartE2EDuration="1m12.947188517s" totalImagesPullingTime="4.021627247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:10.946992455 +0000 UTC m=+72.870917773" watchObservedRunningTime="2026-07-24 15:21:10.947188517 +0000 UTC m=+72.871113891" Jul 24 15:21:10.954018 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:10.953989 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftppt"] Jul 24 15:21:10.956957 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:10.956927 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092e3a8a_1471_4f54_ab2d_9653c2638d65.slice/crio-c015fa1939e1a44d111db437a6aa70f424b98db9a42d7185b9cda2c4e55d229d WatchSource:0}: Error finding container c015fa1939e1a44d111db437a6aa70f424b98db9a42d7185b9cda2c4e55d229d: Status 404 returned error can't find the container with id c015fa1939e1a44d111db437a6aa70f424b98db9a42d7185b9cda2c4e55d229d Jul 24 15:21:11.935899 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:11.935853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftppt" event={"ID":"092e3a8a-1471-4f54-ab2d-9653c2638d65","Type":"ContainerStarted","Data":"c015fa1939e1a44d111db437a6aa70f424b98db9a42d7185b9cda2c4e55d229d"} Jul 24 15:21:16.957094 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:16.957025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftppt" event={"ID":"092e3a8a-1471-4f54-ab2d-9653c2638d65","Type":"ContainerStarted","Data":"d2d579aaef3e371ea64499106c13975fa97cca2b95d749b137e033bde7b01041"} Jul 24 15:21:19.970209 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:19.970167 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftppt" event={"ID":"092e3a8a-1471-4f54-ab2d-9653c2638d65","Type":"ContainerStarted","Data":"bad93aba8f2b4005ae6895a6e8655e0f02f099b5b9a50ba876fc69e4438809af"} Jul 24 15:21:23.929436 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.929397 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d69kr"] Jul 24 15:21:23.936129 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.936104 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.940276 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.940256 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 15:21:23.941308 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.941288 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 15:21:23.941458 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.941442 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 15:21:23.941558 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.941443 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 15:21:23.941558 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.941540 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cdzmv\"" Jul 24 15:21:23.942700 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.942682 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 15:21:23.943746 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.943730 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 15:21:23.986711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.986679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftppt" event={"ID":"092e3a8a-1471-4f54-ab2d-9653c2638d65","Type":"ContainerStarted","Data":"52ef79e1150b59b576a91d4e0fc8f619c6744d75e5d9755ff85c8aacb4f6d3f6"} Jul 24 15:21:23.988138 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-root\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988190 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-tls\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988190 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-sys\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988190 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988181 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988298 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988211 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czzp6\" (UniqueName: \"kubernetes.io/projected/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-kube-api-access-czzp6\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988298 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988259 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-metrics-client-ca\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988356 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-wtmp\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988356 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:23.988427 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:23.988366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-textfile\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.006887 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.006845 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ftppt" podStartSLOduration=34.040392077 podStartE2EDuration="46.006831007s" totalImagesPullingTime="11.96643893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:20:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:24.006187103 +0000 UTC m=+85.930112413" watchObservedRunningTime="2026-07-24 15:21:24.006831007 +0000 UTC m=+85.930756314" Jul 24 15:21:24.088802 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088774 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-wtmp\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-textfile\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088869 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-root\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-tls\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088916 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-sys\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-sys\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.088991 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088968 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-root\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.089353 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.088995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-wtmp\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.089353 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.089018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.089353 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.089066 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czzp6\" (UniqueName: \"kubernetes.io/projected/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-kube-api-access-czzp6\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.089353 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.089095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-metrics-client-ca\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.089353 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.089248 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-textfile\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.089591 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.089570 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-accelerators-collector-config\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.090078 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.090056 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-metrics-client-ca\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.091359 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.091331 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.091483 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.091475 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-node-exporter-tls\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.098227 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.098195 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czzp6\" (UniqueName: \"kubernetes.io/projected/9fb4cadc-d7a3-42a0-a220-9cd17b3088f5-kube-api-access-czzp6\") pod \"node-exporter-d69kr\" (UID: \"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5\") " pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.244683 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.244637 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d69kr" Jul 24 15:21:24.254778 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:24.254752 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb4cadc_d7a3_42a0_a220_9cd17b3088f5.slice/crio-228aa38f484996146ec26002393e306afa23f4052c9c9c284d935e949a0f1ff9 WatchSource:0}: Error finding container 228aa38f484996146ec26002393e306afa23f4052c9c9c284d935e949a0f1ff9: Status 404 returned error can't find the container with id 228aa38f484996146ec26002393e306afa23f4052c9c9c284d935e949a0f1ff9 Jul 24 15:21:24.967159 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.967128 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:21:24.970790 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.970768 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:24.974688 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.974664 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 15:21:24.975178 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975159 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 15:21:24.975267 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975232 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 15:21:24.975267 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975241 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 15:21:24.975622 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975527 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 15:21:24.975754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975663 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 15:21:24.975754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975696 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8fb7j\"" Jul 24 15:21:24.975754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975746 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 15:21:24.975996 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975976 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 15:21:24.976120 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.975977 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 15:21:24.991877 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.991853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d69kr" event={"ID":"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5","Type":"ContainerStarted","Data":"228aa38f484996146ec26002393e306afa23f4052c9c9c284d935e949a0f1ff9"} Jul 24 15:21:24.998271 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:24.998249 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:21:25.095969 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.095936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.095969 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.095971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096222 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-web-config\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096222 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-config-volume\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096222 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096138 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096222 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096173 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-config-out\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjndd\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-kube-api-access-bjndd\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096309 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.096604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.096463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.196823 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.196800 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-config-out\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.196909 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.196834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.196909 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.196856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.196909 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.196872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjndd\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-kube-api-access-bjndd\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.196909 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.196897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.197154 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.196925 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.197291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.197349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.197423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.197450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:21:25.197579 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle podName:57010541-9a40-4c0d-9fc4-a57556690601 nodeName:}" failed. No retries permitted until 2026-07-24 15:21:25.697555139 +0000 UTC m=+87.621480431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "57010541-9a40-4c0d-9fc4-a57556690601") : configmap references non-existent config key: ca-bundle.crt Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.197964 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.198016 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.198056 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-web-config\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.198096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-config-volume\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.198123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:21:25.198242 2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 24 15:21:25.198636 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:21:25.198294 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls podName:57010541-9a40-4c0d-9fc4-a57556690601 nodeName:}" failed. No retries permitted until 2026-07-24 15:21:25.698273423 +0000 UTC m=+87.622198710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "57010541-9a40-4c0d-9fc4-a57556690601") : secret "alertmanager-main-tls" not found Jul 24 15:21:25.200109 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.200084 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.200231 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.200191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.200295 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.200259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.200490 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.200466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-config-out\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.200594 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.200580 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.201401 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.201386 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-web-config\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.201529 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.201513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.201758 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.201743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-config-volume\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.214008 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.213978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjndd\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-kube-api-access-bjndd\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.702317 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.702278 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.702497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.702334 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.703143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.703075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.704819 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.704797 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.882107 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.882073 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:21:25.996915 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.996884 2564 generic.go:356] "Generic (PLEG): container finished" podID="9fb4cadc-d7a3-42a0-a220-9cd17b3088f5" containerID="d423a1c2b631a46aa76847d60bb97e1505ba482ba7d1963d4ba78071a730003f" exitCode=0 Jul 24 15:21:25.997320 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:25.996969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d69kr" event={"ID":"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5","Type":"ContainerDied","Data":"d423a1c2b631a46aa76847d60bb97e1505ba482ba7d1963d4ba78071a730003f"} Jul 24 15:21:26.021332 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:26.021298 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:21:26.031745 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:26.031720 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57010541_9a40_4c0d_9fc4_a57556690601.slice/crio-80549c11bb8734584486df177919fe3ec071929a00d85a13998438e45cef854c WatchSource:0}: Error finding container 80549c11bb8734584486df177919fe3ec071929a00d85a13998438e45cef854c: Status 404 returned error can't find the container with id 80549c11bb8734584486df177919fe3ec071929a00d85a13998438e45cef854c Jul 24 15:21:27.004885 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:27.004840 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d69kr" event={"ID":"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5","Type":"ContainerStarted","Data":"937012590c042c486e46eb79ffe078cfabafd3e502bb1ffd6f7292678635a64b"} Jul 24 15:21:27.005374 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:27.004905 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d69kr" event={"ID":"9fb4cadc-d7a3-42a0-a220-9cd17b3088f5","Type":"ContainerStarted","Data":"2daac86fa069bf5480d56a74784f4879d3598700262f15b22923bb3b9a9c9696"} Jul 24 15:21:27.006444 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:27.006362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"80549c11bb8734584486df177919fe3ec071929a00d85a13998438e45cef854c"} Jul 24 15:21:27.026629 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:27.026561 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d69kr" podStartSLOduration=3.055618514 podStartE2EDuration="4.02654045s" totalImagesPullingTime="899.945141ms" totalInitContainerRuntime="70.976795ms" isStatefulPod=true podCreationTimestamp="2026-07-24 15:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:27.02547056 +0000 UTC m=+88.949395870" watchObservedRunningTime="2026-07-24 15:21:27.02654045 +0000 UTC m=+88.950465759" Jul 24 15:21:28.011665 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:28.011618 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0" exitCode=0 Jul 24 15:21:28.012030 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:28.011682 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0"} Jul 24 15:21:30.023161 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.023125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153"} Jul 24 15:21:30.023544 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.023198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6"} Jul 24 15:21:30.023544 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.023229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4"} Jul 24 15:21:30.023544 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.023258 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff"} Jul 24 15:21:30.023544 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.023281 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53"} Jul 24 15:21:30.191326 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.191295 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:21:30.194211 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.194195 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.196947 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.196924 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 15:21:30.197065 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.196922 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 15:21:30.197262 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.197237 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 15:21:30.197369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.197314 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 15:21:30.197369 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.197320 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1braaug5oqvcv\"" Jul 24 15:21:30.197542 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.197526 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 15:21:30.198247 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.198232 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 15:21:30.198296 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.198272 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 15:21:30.198585 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.198569 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 15:21:30.198678 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.198573 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 15:21:30.200245 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.200226 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 15:21:30.200547 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.200470 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 15:21:30.200547 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.200532 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b2z4x\"" Jul 24 15:21:30.200735 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.200554 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 15:21:30.200735 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.200566 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 15:21:30.200917 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.200901 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 15:21:30.201023 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.201005 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 15:21:30.215845 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.215816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:21:30.243290 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243290 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243290 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243394 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243454 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243471 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzsm5\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-kube-api-access-zzsm5\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243619 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-web-config\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-config\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243711 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243993 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243993 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243993 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.243993 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.243778 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-config-out\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.344853 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.344816 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.344853 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.344850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345103 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.344867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzsm5\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-kube-api-access-zzsm5\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345103 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.344885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345103 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345007 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345103 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345103 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-web-config\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-config\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345278 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345298 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345345 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-config-out\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345397 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345727 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345801 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.345851 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345816 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.346187 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.345615 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.346860 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.346837 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.347232 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.347203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.348783 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.348738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.349049 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.349009 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.349686 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.349643 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.349813 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.349781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-web-config\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.349999 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.349978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.350177 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.350154 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.350263 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.350116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.350432 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.350413 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-config\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.350621 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.350604 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.350865 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.350846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.350945 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.350857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-config-out\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.351640 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.351614 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.351826 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.351811 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.353586 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.353572 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzsm5\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-kube-api-access-zzsm5\") pod \"prometheus-k8s-0\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.504464 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.504383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:21:30.661806 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:30.661772 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:21:30.773181 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:21:30.773104 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20224a53_cefd_43e4_88ac_898ab75debec.slice/crio-d2dbed0237eac147382a50768dde6f2d4174b07b9a9a2378b1a2d3d0868b7371 WatchSource:0}: Error finding container d2dbed0237eac147382a50768dde6f2d4174b07b9a9a2378b1a2d3d0868b7371: Status 404 returned error can't find the container with id d2dbed0237eac147382a50768dde6f2d4174b07b9a9a2378b1a2d3d0868b7371 Jul 24 15:21:31.028061 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:31.027972 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" exitCode=0 Jul 24 15:21:31.028577 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:31.028069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} Jul 24 15:21:31.028577 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:31.028154 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"d2dbed0237eac147382a50768dde6f2d4174b07b9a9a2378b1a2d3d0868b7371"} Jul 24 15:21:31.031777 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:31.031747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerStarted","Data":"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706"} Jul 24 15:21:31.085104 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:31.085058 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.118645766 podStartE2EDuration="7.085042058s" totalImagesPullingTime="3.948836516s" totalInitContainerRuntime="17.559776ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:21:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:31.083252215 +0000 UTC m=+93.007177575" watchObservedRunningTime="2026-07-24 15:21:31.085042058 +0000 UTC m=+93.008967367" Jul 24 15:21:35.052949 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:35.052905 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} Jul 24 15:21:35.053356 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:35.052983 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} Jul 24 15:21:36.066162 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:36.066127 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} Jul 24 15:21:36.066518 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:36.066220 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} Jul 24 15:21:37.073350 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:37.073308 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} Jul 24 15:21:37.073768 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:37.073393 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerStarted","Data":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} Jul 24 15:21:37.113713 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:37.113629 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.358323516 podStartE2EDuration="7.113607878s" totalImagesPullingTime="4.734365106s" totalInitContainerRuntime="20.919256ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:21:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:21:37.111026366 +0000 UTC m=+99.034951723" watchObservedRunningTime="2026-07-24 15:21:37.113607878 +0000 UTC m=+99.037533188" Jul 24 15:21:37.918828 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:37.918799 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4sbjk" Jul 24 15:21:40.505360 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:21:40.505323 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:30.505292 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:30.505244 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:30.525372 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:30.525348 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:31.285870 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:31.285842 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:44.223020 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.222989 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:22:44.223405 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.223268 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="alertmanager" containerID="cri-o://313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53" gracePeriod=120 Jul 24 15:22:44.223405 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.223326 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-metric" containerID="cri-o://eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153" gracePeriod=120 Jul 24 15:22:44.223510 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.223389 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="prom-label-proxy" containerID="cri-o://fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706" gracePeriod=120 Jul 24 15:22:44.223510 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.223427 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy" containerID="cri-o://7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6" gracePeriod=120 Jul 24 15:22:44.223510 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.223398 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-web" containerID="cri-o://f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4" gracePeriod=120 Jul 24 15:22:44.223633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:44.223407 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="config-reloader" containerID="cri-o://214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff" gracePeriod=120 Jul 24 15:22:45.319800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319768 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706" exitCode=0 Jul 24 15:22:45.319800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319792 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6" exitCode=0 Jul 24 15:22:45.319800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319798 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff" exitCode=0 Jul 24 15:22:45.319800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319803 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53" exitCode=0 Jul 24 15:22:45.320263 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319839 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706"} Jul 24 15:22:45.320263 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319905 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6"} Jul 24 15:22:45.320263 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff"} Jul 24 15:22:45.320263 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.319954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53"} Jul 24 15:22:45.463627 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.463601 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:45.588834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588741 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjndd\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-kube-api-access-bjndd\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.588834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588789 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.588834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588818 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589088 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588855 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-web\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589088 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588880 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-cluster-tls-config\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589088 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588906 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589330 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.588935 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-config-volume\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.589468 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-config-out\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589640 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.589624 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-web-config\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589836 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.589818 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-metrics-client-ca\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.589948 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.589935 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-tls-assets\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.590231 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.590130 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-main-db\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.590410 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.590393 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-metric\") pod \"57010541-9a40-4c0d-9fc4-a57556690601\" (UID: \"57010541-9a40-4c0d-9fc4-a57556690601\") " Jul 24 15:22:45.592034 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.589698 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:45.592034 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.591942 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:45.592278 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.592187 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:22:45.594701 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.592861 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.594701 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.593423 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.594701 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.594345 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-config-out" (OuterVolumeSpecName: "config-out") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:22:45.597922 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.597897 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.598039 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.597928 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.598449 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.598411 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-config-volume" (OuterVolumeSpecName: "config-volume") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.598915 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.598893 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:22:45.599429 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.599410 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-kube-api-access-bjndd" (OuterVolumeSpecName: "kube-api-access-bjndd") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "kube-api-access-bjndd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:22:45.600596 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.600569 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.607616 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.607597 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-web-config" (OuterVolumeSpecName: "web-config") pod "57010541-9a40-4c0d-9fc4-a57556690601" (UID: "57010541-9a40-4c0d-9fc4-a57556690601"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:45.691752 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691721 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-tls-assets\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691752 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691745 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-main-db\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691752 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691757 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691768 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjndd\" (UniqueName: \"kubernetes.io/projected/57010541-9a40-4c0d-9fc4-a57556690601-kube-api-access-bjndd\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691776 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-main-tls\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691786 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691794 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691803 2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-cluster-tls-config\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691811 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691818 2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-config-volume\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691826 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57010541-9a40-4c0d-9fc4-a57556690601-config-out\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691853 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57010541-9a40-4c0d-9fc4-a57556690601-web-config\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:45.691952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:45.691861 2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57010541-9a40-4c0d-9fc4-a57556690601-metrics-client-ca\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:46.326480 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326441 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153" exitCode=0 Jul 24 15:22:46.326480 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326473 2564 generic.go:356] "Generic (PLEG): container finished" podID="57010541-9a40-4c0d-9fc4-a57556690601" containerID="f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4" exitCode=0 Jul 24 15:22:46.326957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326519 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153"} Jul 24 15:22:46.326957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326541 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.326957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326622 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4"} Jul 24 15:22:46.326957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57010541-9a40-4c0d-9fc4-a57556690601","Type":"ContainerDied","Data":"80549c11bb8734584486df177919fe3ec071929a00d85a13998438e45cef854c"} Jul 24 15:22:46.326957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.326679 2564 scope.go:117] "RemoveContainer" containerID="fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706" Jul 24 15:22:46.335496 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.335439 2564 scope.go:117] "RemoveContainer" containerID="eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153" Jul 24 15:22:46.343442 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.343423 2564 scope.go:117] "RemoveContainer" containerID="7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6" Jul 24 15:22:46.350029 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.349947 2564 scope.go:117] "RemoveContainer" containerID="f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4" Jul 24 15:22:46.353683 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.353032 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:22:46.376990 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.376955 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:22:46.379206 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.379192 2564 scope.go:117] "RemoveContainer" containerID="214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff" Jul 24 15:22:46.386691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.386674 2564 scope.go:117] "RemoveContainer" containerID="313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53" Jul 24 15:22:46.390033 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390013 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:22:46.390279 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390267 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-web" Jul 24 15:22:46.390322 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390283 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-web" Jul 24 15:22:46.390322 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390296 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-metric" Jul 24 15:22:46.390322 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390302 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-metric" Jul 24 15:22:46.390322 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390315 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="prom-label-proxy" Jul 24 15:22:46.390322 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390320 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="prom-label-proxy" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390327 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="init-config-reloader" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390333 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="init-config-reloader" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390341 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="alertmanager" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390346 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="alertmanager" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390353 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="config-reloader" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390362 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="config-reloader" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390375 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390380 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390420 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="alertmanager" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390429 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-web" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390436 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390444 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="prom-label-proxy" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390452 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="config-reloader" Jul 24 15:22:46.390486 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.390457 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57010541-9a40-4c0d-9fc4-a57556690601" containerName="kube-rbac-proxy-metric" Jul 24 15:22:46.394208 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.394189 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.394734 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.394714 2564 scope.go:117] "RemoveContainer" containerID="25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0" Jul 24 15:22:46.397022 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397004 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8fb7j\"" Jul 24 15:22:46.397507 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397489 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 15:22:46.397592 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397511 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 15:22:46.397684 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397629 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 15:22:46.397752 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397638 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 15:22:46.397866 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397850 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 15:22:46.397933 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397869 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 15:22:46.397933 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.397914 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 15:22:46.398087 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.398073 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 15:22:46.402940 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.402920 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 15:22:46.404354 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.404342 2564 scope.go:117] "RemoveContainer" containerID="fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706" Jul 24 15:22:46.404622 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.404600 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706\": container with ID starting with fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706 not found: ID does not exist" containerID="fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706" Jul 24 15:22:46.404694 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.404635 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706"} err="failed to get container status \"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706\": rpc error: code = NotFound desc = could not find container \"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706\": container with ID starting with fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706 not found: ID does not exist" Jul 24 15:22:46.404766 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.404692 2564 scope.go:117] "RemoveContainer" containerID="eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153" Jul 24 15:22:46.404927 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.404901 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153\": container with ID starting with eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153 not found: ID does not exist" containerID="eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153" Jul 24 15:22:46.404986 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.404935 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153"} err="failed to get container status \"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153\": rpc error: code = NotFound desc = could not find container \"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153\": container with ID starting with eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153 not found: ID does not exist" Jul 24 15:22:46.404986 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.404954 2564 scope.go:117] "RemoveContainer" containerID="7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6" Jul 24 15:22:46.405172 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.405157 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6\": container with ID starting with 7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6 not found: ID does not exist" containerID="7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6" Jul 24 15:22:46.405225 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405178 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6"} err="failed to get container status \"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6\": rpc error: code = NotFound desc = could not find container \"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6\": container with ID starting with 7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6 not found: ID does not exist" Jul 24 15:22:46.405225 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405192 2564 scope.go:117] "RemoveContainer" containerID="f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4" Jul 24 15:22:46.405397 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.405382 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4\": container with ID starting with f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4 not found: ID does not exist" containerID="f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4" Jul 24 15:22:46.405433 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405401 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4"} err="failed to get container status \"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4\": rpc error: code = NotFound desc = could not find container \"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4\": container with ID starting with f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4 not found: ID does not exist" Jul 24 15:22:46.405433 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405413 2564 scope.go:117] "RemoveContainer" containerID="214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff" Jul 24 15:22:46.405612 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.405595 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff\": container with ID starting with 214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff not found: ID does not exist" containerID="214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff" Jul 24 15:22:46.405699 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405620 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff"} err="failed to get container status \"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff\": rpc error: code = NotFound desc = could not find container \"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff\": container with ID starting with 214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff not found: ID does not exist" Jul 24 15:22:46.405699 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405641 2564 scope.go:117] "RemoveContainer" containerID="313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53" Jul 24 15:22:46.405897 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.405878 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53\": container with ID starting with 313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53 not found: ID does not exist" containerID="313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53" Jul 24 15:22:46.405957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405905 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53"} err="failed to get container status \"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53\": rpc error: code = NotFound desc = could not find container \"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53\": container with ID starting with 313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53 not found: ID does not exist" Jul 24 15:22:46.405957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.405924 2564 scope.go:117] "RemoveContainer" containerID="25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0" Jul 24 15:22:46.406130 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:46.406114 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0\": container with ID starting with 25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0 not found: ID does not exist" containerID="25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0" Jul 24 15:22:46.406189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406135 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0"} err="failed to get container status \"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0\": rpc error: code = NotFound desc = could not find container \"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0\": container with ID starting with 25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0 not found: ID does not exist" Jul 24 15:22:46.406189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406153 2564 scope.go:117] "RemoveContainer" containerID="fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706" Jul 24 15:22:46.406390 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406373 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706"} err="failed to get container status \"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706\": rpc error: code = NotFound desc = could not find container \"fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706\": container with ID starting with fdfb3fb44c3f12824ea203f961b7c02032d785950569e06e7b7470547efb5706 not found: ID does not exist" Jul 24 15:22:46.406455 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406391 2564 scope.go:117] "RemoveContainer" containerID="eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153" Jul 24 15:22:46.406598 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406582 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153"} err="failed to get container status \"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153\": rpc error: code = NotFound desc = could not find container \"eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153\": container with ID starting with eeca3e7b4c44b2242564d25affeaa3f0fa6f2c0f341f4bda5d3a6db652747153 not found: ID does not exist" Jul 24 15:22:46.406705 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406599 2564 scope.go:117] "RemoveContainer" containerID="7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6" Jul 24 15:22:46.406823 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406807 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6"} err="failed to get container status \"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6\": rpc error: code = NotFound desc = could not find container \"7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6\": container with ID starting with 7b4c4ae4299a757a0b77d11ad977f191841d4a95e23085938914dc83b315d6d6 not found: ID does not exist" Jul 24 15:22:46.406882 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.406824 2564 scope.go:117] "RemoveContainer" containerID="f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4" Jul 24 15:22:46.407057 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407026 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4"} err="failed to get container status \"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4\": rpc error: code = NotFound desc = could not find container \"f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4\": container with ID starting with f38349c2556dc8cebfeca78439a6db47f631553ee95f1b3825c37a9545e80ef4 not found: ID does not exist" Jul 24 15:22:46.407104 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407058 2564 scope.go:117] "RemoveContainer" containerID="214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff" Jul 24 15:22:46.407285 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407264 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff"} err="failed to get container status \"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff\": rpc error: code = NotFound desc = could not find container \"214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff\": container with ID starting with 214de5df03499b92050eac13b699644f43000ad3b35424cb492e7684c942a2ff not found: ID does not exist" Jul 24 15:22:46.407329 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407286 2564 scope.go:117] "RemoveContainer" containerID="313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53" Jul 24 15:22:46.407493 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407476 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53"} err="failed to get container status \"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53\": rpc error: code = NotFound desc = could not find container \"313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53\": container with ID starting with 313d7f67bad878d86c7f97166ab0f3fc29003ee1e97abde5bf0a3c8193854c53 not found: ID does not exist" Jul 24 15:22:46.407558 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407494 2564 scope.go:117] "RemoveContainer" containerID="25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0" Jul 24 15:22:46.407753 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.407735 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0"} err="failed to get container status \"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0\": rpc error: code = NotFound desc = could not find container \"25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0\": container with ID starting with 25ecf96549678e0de00a469e00ef9a171827f5c34edec4e7e3a2679b8695dcb0 not found: ID does not exist" Jul 24 15:22:46.412620 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.412601 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:22:46.497635 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497602 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4daa1da-d627-412c-9f76-08439cacd9c6-config-out\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.497805 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.497805 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4daa1da-d627-412c-9f76-08439cacd9c6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.497805 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497726 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.497805 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4daa1da-d627-412c-9f76-08439cacd9c6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.497805 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4daa1da-d627-412c-9f76-08439cacd9c6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjgjz\" (UniqueName: \"kubernetes.io/projected/f4daa1da-d627-412c-9f76-08439cacd9c6-kube-api-access-xjgjz\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497875 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497893 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4daa1da-d627-412c-9f76-08439cacd9c6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497913 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497938 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.497993 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-web-config\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.498189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.498045 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.532724 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.532620 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57010541-9a40-4c0d-9fc4-a57556690601" path="/var/lib/kubelet/pods/57010541-9a40-4c0d-9fc4-a57556690601/volumes" Jul 24 15:22:46.599416 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599416 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4daa1da-d627-412c-9f76-08439cacd9c6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4daa1da-d627-412c-9f76-08439cacd9c6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4daa1da-d627-412c-9f76-08439cacd9c6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599568 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjgjz\" (UniqueName: \"kubernetes.io/projected/f4daa1da-d627-412c-9f76-08439cacd9c6-kube-api-access-xjgjz\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599608 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.599633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4daa1da-d627-412c-9f76-08439cacd9c6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600018 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.599998 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4daa1da-d627-412c-9f76-08439cacd9c6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600083 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600137 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600137 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-web-config\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4daa1da-d627-412c-9f76-08439cacd9c6-config-out\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600865 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4daa1da-d627-412c-9f76-08439cacd9c6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.600865 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.600781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4daa1da-d627-412c-9f76-08439cacd9c6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.603226 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.602826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.603226 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.603056 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.603226 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.603189 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4daa1da-d627-412c-9f76-08439cacd9c6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.603452 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.603359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.603572 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.603552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.603789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.603765 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.604126 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.604111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-web-config\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.604299 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.604278 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4daa1da-d627-412c-9f76-08439cacd9c6-config-out\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.604633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.604617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4daa1da-d627-412c-9f76-08439cacd9c6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.608610 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.608592 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjgjz\" (UniqueName: \"kubernetes.io/projected/f4daa1da-d627-412c-9f76-08439cacd9c6-kube-api-access-xjgjz\") pod \"alertmanager-main-0\" (UID: \"f4daa1da-d627-412c-9f76-08439cacd9c6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.704820 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.704783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:22:46.844746 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:46.844717 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:22:46.846874 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:22:46.846850 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4daa1da_d627_412c_9f76_08439cacd9c6.slice/crio-26b3f293904c94005d0a87b2e6ea9058972fa10c51d07dfdd074c8e4023ce186 WatchSource:0}: Error finding container 26b3f293904c94005d0a87b2e6ea9058972fa10c51d07dfdd074c8e4023ce186: Status 404 returned error can't find the container with id 26b3f293904c94005d0a87b2e6ea9058972fa10c51d07dfdd074c8e4023ce186 Jul 24 15:22:47.331560 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:47.331526 2564 generic.go:356] "Generic (PLEG): container finished" podID="f4daa1da-d627-412c-9f76-08439cacd9c6" containerID="b3bb8362d412f1b3d4075959509ac03f6614531ef2709a26ef8058e48a246bda" exitCode=0 Jul 24 15:22:47.332051 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:47.331626 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerDied","Data":"b3bb8362d412f1b3d4075959509ac03f6614531ef2709a26ef8058e48a246bda"} Jul 24 15:22:47.332051 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:47.331740 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"26b3f293904c94005d0a87b2e6ea9058972fa10c51d07dfdd074c8e4023ce186"} Jul 24 15:22:48.344168 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.344128 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"7dddb464cf254c04f21d91293fef3c11eb79c8748500a76c996e57c1c367b3e0"} Jul 24 15:22:48.344587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.344241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"2049816e72a5cc2418e339e74bd0789651bd1f6643a4b251a17c4a5070432908"} Jul 24 15:22:48.344587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.344286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"9ef648b393b3853bd48708d8c75e7368200a63d25e28237fe4d9f0fa4941b47c"} Jul 24 15:22:48.344587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.344322 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"f16f1f6a37aec82f463dbc0550e961ef92f85b8a859579e82b5fd266e1db4e6b"} Jul 24 15:22:48.344587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.344358 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"0d0c3afbd2e7bf4b9045faba1000439d0c7ef67324cca6ba303ef533f37cb568"} Jul 24 15:22:48.344587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.344410 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4daa1da-d627-412c-9f76-08439cacd9c6","Type":"ContainerStarted","Data":"2525643d3edc51ef306a87ed07d760c94a39cdf74327d8a5bce169495db4a120"} Jul 24 15:22:48.369856 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.369794 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.350585266 podStartE2EDuration="2.369773227s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.187961ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:22:48.368472678 +0000 UTC m=+170.292398008" watchObservedRunningTime="2026-07-24 15:22:48.369773227 +0000 UTC m=+170.293698536" Jul 24 15:22:48.581044 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581011 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:22:48.581402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581364 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="prometheus" containerID="cri-o://525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" gracePeriod=600 Jul 24 15:22:48.581402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581385 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy" containerID="cri-o://71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" gracePeriod=600 Jul 24 15:22:48.581638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581403 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-thanos" containerID="cri-o://49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" gracePeriod=600 Jul 24 15:22:48.581638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581431 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="config-reloader" containerID="cri-o://014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" gracePeriod=600 Jul 24 15:22:48.581638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581431 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="thanos-sidecar" containerID="cri-o://30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" gracePeriod=600 Jul 24 15:22:48.581638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.581537 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-web" containerID="cri-o://b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" gracePeriod=600 Jul 24 15:22:48.810188 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.810165 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:48.818535 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818515 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-config\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818546 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-web-config\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818572 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-trusted-ca-bundle\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818594 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzsm5\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-kube-api-access-zzsm5\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818610 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-serving-certs-ca-bundle\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818632 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-db\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818684 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-1\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818718 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-0\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818739 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-tls-assets\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818776 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-tls\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818825 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-2\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818850 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-kubelet-serving-ca-bundle\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818876 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818904 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-thanos-prometheus-http-client-file\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818930 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.818967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818958 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-metrics-client-ca\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.819380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.818992 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-config-out\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.819380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819001 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.819380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819011 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-grpc-tls\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.819380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819177 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-kube-rbac-proxy\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.819380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819210 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-metrics-client-certs\") pod \"20224a53-cefd-43e4-88ac-898ab75debec\" (UID: \"20224a53-cefd-43e4-88ac-898ab75debec\") " Jul 24 15:22:48.819380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819360 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.819845 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819447 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.819845 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.819464 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.820900 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.820722 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.820900 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.820790 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.820900 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.820833 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.821552 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.821463 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.821907 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.821768 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:22:48.821907 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.821857 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:22:48.822066 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.822005 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-kube-api-access-zzsm5" (OuterVolumeSpecName: "kube-api-access-zzsm5") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "kube-api-access-zzsm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:22:48.822431 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.822325 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-config" (OuterVolumeSpecName: "config") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.822431 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.822408 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.823339 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.823313 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:22:48.823587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.823528 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.823587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.823527 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-config-out" (OuterVolumeSpecName: "config-out") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:22:48.823996 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.823974 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.824128 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.824091 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.824249 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.824221 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.824325 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.824266 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.824946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.824859 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.832314 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.832295 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-web-config" (OuterVolumeSpecName: "web-config") pod "20224a53-cefd-43e4-88ac-898ab75debec" (UID: "20224a53-cefd-43e4-88ac-898ab75debec"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920455 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzsm5\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-kube-api-access-zzsm5\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920485 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-db\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920498 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920511 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920524 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20224a53-cefd-43e4-88ac-898ab75debec-tls-assets\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920536 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920550 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920565 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920579 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920591 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920602 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920615 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20224a53-cefd-43e4-88ac-898ab75debec-configmap-metrics-client-ca\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920627 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20224a53-cefd-43e4-88ac-898ab75debec-config-out\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920638 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-grpc-tls\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920678 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-kube-rbac-proxy\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920690 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-secret-metrics-client-certs\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920702 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-config\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:48.920928 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:48.920713 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20224a53-cefd-43e4-88ac-898ab75debec-web-config\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:22:49.350796 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350757 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" exitCode=0 Jul 24 15:22:49.350796 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350790 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" exitCode=0 Jul 24 15:22:49.350796 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350799 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" exitCode=0 Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350807 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" exitCode=0 Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350815 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" exitCode=0 Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350823 2564 generic.go:356] "Generic (PLEG): container finished" podID="20224a53-cefd-43e4-88ac-898ab75debec" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" exitCode=0 Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350830 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350847 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350895 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350974 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.350998 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} Jul 24 15:22:49.351272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.351022 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20224a53-cefd-43e4-88ac-898ab75debec","Type":"ContainerDied","Data":"d2dbed0237eac147382a50768dde6f2d4174b07b9a9a2378b1a2d3d0868b7371"} Jul 24 15:22:49.359598 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.359585 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.366818 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.366801 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.373595 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.373578 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.377719 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.377694 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:22:49.380763 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.380747 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.385332 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.385308 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:22:49.388142 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.388125 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.395609 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.395591 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.402548 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.402529 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.402839 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.402807 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.402927 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.402853 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} err="failed to get container status \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" Jul 24 15:22:49.402927 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.402879 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.403141 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.403124 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.403178 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403149 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} err="failed to get container status \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" Jul 24 15:22:49.403178 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403166 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.403399 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.403383 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.403443 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403404 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} err="failed to get container status \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" Jul 24 15:22:49.403443 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403416 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.403678 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.403659 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.403719 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403683 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} err="failed to get container status \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" Jul 24 15:22:49.403719 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403706 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.403921 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.403900 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.403995 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403924 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} err="failed to get container status \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" Jul 24 15:22:49.403995 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.403939 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.404166 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.404152 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.404213 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404169 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} err="failed to get container status \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" Jul 24 15:22:49.404213 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404180 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.404410 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:22:49.404393 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.404450 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404413 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} err="failed to get container status \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" Jul 24 15:22:49.404450 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404428 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.404733 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404715 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} err="failed to get container status \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" Jul 24 15:22:49.404733 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404733 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.404958 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404921 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} err="failed to get container status \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" Jul 24 15:22:49.404958 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.404958 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.405249 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.405231 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} err="failed to get container status \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" Jul 24 15:22:49.405318 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.405250 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.405520 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.405502 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} err="failed to get container status \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" Jul 24 15:22:49.405572 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.405520 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.405760 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.405739 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} err="failed to get container status \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" Jul 24 15:22:49.405760 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.405759 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.406018 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406001 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} err="failed to get container status \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" Jul 24 15:22:49.406075 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406019 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.406260 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406240 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} err="failed to get container status \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" Jul 24 15:22:49.406314 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406261 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.406494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406478 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} err="failed to get container status \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" Jul 24 15:22:49.406494 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406494 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.406722 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406695 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} err="failed to get container status \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" Jul 24 15:22:49.406760 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406723 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.406934 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406918 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} err="failed to get container status \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" Jul 24 15:22:49.406934 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.406934 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.407125 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407111 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} err="failed to get container status \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" Jul 24 15:22:49.407160 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407127 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.407327 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407308 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} err="failed to get container status \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" Jul 24 15:22:49.407366 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407329 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.407511 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407496 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} err="failed to get container status \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" Jul 24 15:22:49.407546 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407514 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.407819 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407801 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} err="failed to get container status \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" Jul 24 15:22:49.407865 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.407820 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.408033 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408015 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} err="failed to get container status \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" Jul 24 15:22:49.408085 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408033 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.408233 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408217 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} err="failed to get container status \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" Jul 24 15:22:49.408269 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408233 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.408417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408401 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} err="failed to get container status \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" Jul 24 15:22:49.408468 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408418 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.408595 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408579 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} err="failed to get container status \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" Jul 24 15:22:49.408641 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408595 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.408825 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408809 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} err="failed to get container status \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" Jul 24 15:22:49.408894 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.408825 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.409042 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409025 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} err="failed to get container status \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" Jul 24 15:22:49.409087 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409041 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.409265 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409246 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} err="failed to get container status \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" Jul 24 15:22:49.409307 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409266 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.409483 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409460 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} err="failed to get container status \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" Jul 24 15:22:49.409519 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409484 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.409694 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409674 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} err="failed to get container status \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" Jul 24 15:22:49.409741 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409697 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.409935 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409913 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} err="failed to get container status \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" Jul 24 15:22:49.409980 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.409937 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.410145 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410128 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} err="failed to get container status \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" Jul 24 15:22:49.410190 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410146 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.410473 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410455 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} err="failed to get container status \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" Jul 24 15:22:49.410546 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410476 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.410705 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410689 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} err="failed to get container status \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" Jul 24 15:22:49.410752 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410706 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.410882 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410867 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} err="failed to get container status \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" Jul 24 15:22:49.410933 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.410882 2564 scope.go:117] "RemoveContainer" containerID="49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285" Jul 24 15:22:49.411051 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411036 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285"} err="failed to get container status \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": rpc error: code = NotFound desc = could not find container \"49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285\": container with ID starting with 49a71c452755d574369bffb9384686010311dee1803938367bb2df9d4c1a7285 not found: ID does not exist" Jul 24 15:22:49.411099 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411052 2564 scope.go:117] "RemoveContainer" containerID="71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d" Jul 24 15:22:49.411216 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411198 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d"} err="failed to get container status \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": rpc error: code = NotFound desc = could not find container \"71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d\": container with ID starting with 71250bdd71a8684c4046a65f949c64c22d0231fc06b304e7337e0bee978b328d not found: ID does not exist" Jul 24 15:22:49.411265 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411217 2564 scope.go:117] "RemoveContainer" containerID="b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c" Jul 24 15:22:49.411441 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411426 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c"} err="failed to get container status \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": rpc error: code = NotFound desc = could not find container \"b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c\": container with ID starting with b4425661384cf3cbd5ca2edc3771a706cb889b30854441aa6da48a18e65daf6c not found: ID does not exist" Jul 24 15:22:49.411483 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411442 2564 scope.go:117] "RemoveContainer" containerID="30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9" Jul 24 15:22:49.411660 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411630 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9"} err="failed to get container status \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": rpc error: code = NotFound desc = could not find container \"30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9\": container with ID starting with 30b52eb39e697ede4170cf851f91e8c0926618ea0aa22e5efe65e5b5ae07e1f9 not found: ID does not exist" Jul 24 15:22:49.411709 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411667 2564 scope.go:117] "RemoveContainer" containerID="014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39" Jul 24 15:22:49.411880 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411863 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39"} err="failed to get container status \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": rpc error: code = NotFound desc = could not find container \"014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39\": container with ID starting with 014d8fd5b3391fb48d0f4e7189aafdd7f12389d7e4a296443c526409f3839e39 not found: ID does not exist" Jul 24 15:22:49.411936 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.411883 2564 scope.go:117] "RemoveContainer" containerID="525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27" Jul 24 15:22:49.412312 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.412177 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27"} err="failed to get container status \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": rpc error: code = NotFound desc = could not find container \"525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27\": container with ID starting with 525f36d7c10bc6942694d1fd8c9b640c25f4ec75c4ca2620e6eb0ee572101f27 not found: ID does not exist" Jul 24 15:22:49.412312 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.412221 2564 scope.go:117] "RemoveContainer" containerID="7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9" Jul 24 15:22:49.412626 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.412597 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9"} err="failed to get container status \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": rpc error: code = NotFound desc = could not find container \"7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9\": container with ID starting with 7803f7fe983e7c572f997349ef868a2e1996cd73f147a5e494a35465866e93c9 not found: ID does not exist" Jul 24 15:22:49.414086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414069 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:22:49.414345 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414333 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-web" Jul 24 15:22:49.414379 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414348 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-web" Jul 24 15:22:49.414379 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414363 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-thanos" Jul 24 15:22:49.414379 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414368 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-thanos" Jul 24 15:22:49.414379 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414375 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="init-config-reloader" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414380 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="init-config-reloader" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414386 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="config-reloader" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414391 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="config-reloader" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414396 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="thanos-sidecar" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414401 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="thanos-sidecar" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414407 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414412 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414423 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="prometheus" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414427 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="prometheus" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414468 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414475 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="config-reloader" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414482 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="thanos-sidecar" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414487 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="prometheus" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414493 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-web" Jul 24 15:22:49.414495 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.414498 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20224a53-cefd-43e4-88ac-898ab75debec" containerName="kube-rbac-proxy-thanos" Jul 24 15:22:49.416921 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.416907 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.420482 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.420455 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 15:22:49.421268 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.421245 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 15:22:49.422697 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.421774 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1braaug5oqvcv\"" Jul 24 15:22:49.422697 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.422115 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 15:22:49.422697 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.422228 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 15:22:49.422897 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.422767 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b2z4x\"" Jul 24 15:22:49.422897 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.422779 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 15:22:49.424017 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.423991 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424210 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424194 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424315 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424413 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-web-config\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424501 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424621 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424608 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424852 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424830 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 15:22:49.424975 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424975 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.424975 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.425110 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424981 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 15:22:49.425158 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.425120 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 15:22:49.425200 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.425187 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42hn4\" (UniqueName: \"kubernetes.io/projected/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-kube-api-access-42hn4\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.425280 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.425258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-config\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.425329 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.425291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-config-out\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.425440 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.424439 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 15:22:49.425540 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.425519 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 15:22:49.425875 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.425834 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 15:22:49.426581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426547 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 15:22:49.426694 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.426694 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.426791 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.426862 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426830 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 15:22:49.427406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.427406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.426988 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.427406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.427048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.427406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.427077 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.427406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.427131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.427759 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.427739 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 15:22:49.435533 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.435515 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 15:22:49.444819 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.444799 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:22:49.528470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528433 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-web-config\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528537 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42hn4\" (UniqueName: \"kubernetes.io/projected/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-kube-api-access-42hn4\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528684 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-config\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528702 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-config-out\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528758 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.528814 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528804 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.529402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.529402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.529402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528876 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.529402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.528908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.529402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.529177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.529402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.529306 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.531669 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.531622 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.531993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.532340 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.532486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.532633 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.532674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.532787 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.532956 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.533008 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-web-config\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.533320 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.533395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.533356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.534037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.533699 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.534094 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.534045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.534406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.534378 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.534638 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.534618 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-config-out\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.535073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.535057 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-config\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.535433 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.535411 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.541390 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.541369 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42hn4\" (UniqueName: \"kubernetes.io/projected/cbbbdfb3-5822-4fe9-9507-d3cdc04d1325-kube-api-access-42hn4\") pod \"prometheus-k8s-0\" (UID: \"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.731313 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.731266 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:22:49.871617 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:49.871576 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:22:49.874201 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:22:49.874172 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbbdfb3_5822_4fe9_9507_d3cdc04d1325.slice/crio-3f0a3160f62f97c53c96c9b261d2baf10319260a12823a998844c67e98422baa WatchSource:0}: Error finding container 3f0a3160f62f97c53c96c9b261d2baf10319260a12823a998844c67e98422baa: Status 404 returned error can't find the container with id 3f0a3160f62f97c53c96c9b261d2baf10319260a12823a998844c67e98422baa Jul 24 15:22:50.356982 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:50.356944 2564 generic.go:356] "Generic (PLEG): container finished" podID="cbbbdfb3-5822-4fe9-9507-d3cdc04d1325" containerID="90b8064e8d313774daa9ab037249f53cabf929bd989cddd5498369d258be7f7b" exitCode=0 Jul 24 15:22:50.357484 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:50.357037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerDied","Data":"90b8064e8d313774daa9ab037249f53cabf929bd989cddd5498369d258be7f7b"} Jul 24 15:22:50.357484 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:50.357134 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"3f0a3160f62f97c53c96c9b261d2baf10319260a12823a998844c67e98422baa"} Jul 24 15:22:50.534432 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:50.534400 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20224a53-cefd-43e4-88ac-898ab75debec" path="/var/lib/kubelet/pods/20224a53-cefd-43e4-88ac-898ab75debec/volumes" Jul 24 15:22:51.366908 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.366877 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"372c573a57e36226062b6d2e5a90b8f1d59d45181c9e3ab68a892dada3ab17d5"} Jul 24 15:22:51.367301 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.366964 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"fbd5c1c3ac81d2f6afce1a1b6492e21678f6a968b89bb4c415ade8980d2b62db"} Jul 24 15:22:51.367301 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.366991 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"0a10a11b62c436d0d877b50880a6e523ce1db829f150000472761e3843a97aa4"} Jul 24 15:22:51.367301 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.367014 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"0724b723c474733dc2569c21e27fad4e4a2b7cd10564020240c2f237aa8edbd1"} Jul 24 15:22:51.367301 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.367036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"483c79e93691de3c62b8c0f03e923bcde102ddba30777d50cde56b2e08a340b3"} Jul 24 15:22:51.367301 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.367059 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cbbbdfb3-5822-4fe9-9507-d3cdc04d1325","Type":"ContainerStarted","Data":"78469255cb8c7b1e62a3c9a135b1104318fc29f378a350cac202a4fd5026ce13"} Jul 24 15:22:51.396960 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:51.396898 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.380021159 podStartE2EDuration="2.396879178s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.858019ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:22:51.396109248 +0000 UTC m=+173.320034577" watchObservedRunningTime="2026-07-24 15:22:51.396879178 +0000 UTC m=+173.320804492" Jul 24 15:22:54.732118 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:22:54.732072 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:23:49.731479 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:23:49.731425 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:23:49.748387 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:23:49.747733 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:23:50.599220 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:23:50.599193 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:24:58.438099 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:24:58.438066 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 15:25:56.261952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.261870 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk"] Jul 24 15:25:56.265064 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.265049 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.267672 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.267632 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 15:25:56.267672 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.267643 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 15:25:56.267849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.267632 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7c7tv\"" Jul 24 15:25:56.267849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.267693 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 15:25:56.267955 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.267873 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 15:25:56.289974 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.289946 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk"] Jul 24 15:25:56.319158 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.319117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8571bdb3-1773-450e-bd43-7f32bd7e52e0-manifests\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.319347 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.319253 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8571bdb3-1773-450e-bd43-7f32bd7e52e0-tmp\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.319347 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.319287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8571bdb3-1773-450e-bd43-7f32bd7e52e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.319347 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.319316 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8571bdb3-1773-450e-bd43-7f32bd7e52e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.319347 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.319331 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x97ls\" (UniqueName: \"kubernetes.io/projected/8571bdb3-1773-450e-bd43-7f32bd7e52e0-kube-api-access-x97ls\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.420009 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.419978 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8571bdb3-1773-450e-bd43-7f32bd7e52e0-manifests\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.420168 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.420030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8571bdb3-1773-450e-bd43-7f32bd7e52e0-tmp\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.420168 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.420147 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8571bdb3-1773-450e-bd43-7f32bd7e52e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.420253 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.420191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8571bdb3-1773-450e-bd43-7f32bd7e52e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.420295 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.420250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x97ls\" (UniqueName: \"kubernetes.io/projected/8571bdb3-1773-450e-bd43-7f32bd7e52e0-kube-api-access-x97ls\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.420426 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.420404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8571bdb3-1773-450e-bd43-7f32bd7e52e0-manifests\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.422513 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.422492 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8571bdb3-1773-450e-bd43-7f32bd7e52e0-tmp\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.422604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.422553 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8571bdb3-1773-450e-bd43-7f32bd7e52e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.422749 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.422731 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8571bdb3-1773-450e-bd43-7f32bd7e52e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.431299 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.431281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x97ls\" (UniqueName: \"kubernetes.io/projected/8571bdb3-1773-450e-bd43-7f32bd7e52e0-kube-api-access-x97ls\") pod \"opendatahub-operator-controller-manager-5f445d9dcf-stggk\" (UID: \"8571bdb3-1773-450e-bd43-7f32bd7e52e0\") " pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.575106 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.575021 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:25:56.723174 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.723140 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk"] Jul 24 15:25:56.726290 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:25:56.726266 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8571bdb3_1773_450e_bd43_7f32bd7e52e0.slice/crio-b9a0ea7ff3f64bb586016509fc4bb966384d96b3fbdf5813dea47515ecea4c91 WatchSource:0}: Error finding container b9a0ea7ff3f64bb586016509fc4bb966384d96b3fbdf5813dea47515ecea4c91: Status 404 returned error can't find the container with id b9a0ea7ff3f64bb586016509fc4bb966384d96b3fbdf5813dea47515ecea4c91 Jul 24 15:25:56.728055 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:56.728038 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:25:57.052692 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:25:57.052636 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" event={"ID":"8571bdb3-1773-450e-bd43-7f32bd7e52e0","Type":"ContainerStarted","Data":"b9a0ea7ff3f64bb586016509fc4bb966384d96b3fbdf5813dea47515ecea4c91"} Jul 24 15:26:00.065990 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:00.065894 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" event={"ID":"8571bdb3-1773-450e-bd43-7f32bd7e52e0","Type":"ContainerStarted","Data":"693bc75e8486aa9c5c3993f449a6061b780a7d8614681860727b541a2a7bcd4c"} Jul 24 15:26:01.071671 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:01.071611 2564 generic.go:356] "Generic (PLEG): container finished" podID="8571bdb3-1773-450e-bd43-7f32bd7e52e0" containerID="693bc75e8486aa9c5c3993f449a6061b780a7d8614681860727b541a2a7bcd4c" exitCode=0 Jul 24 15:26:01.072060 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:01.071686 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" event={"ID":"8571bdb3-1773-450e-bd43-7f32bd7e52e0","Type":"ContainerDied","Data":"693bc75e8486aa9c5c3993f449a6061b780a7d8614681860727b541a2a7bcd4c"} Jul 24 15:26:02.077446 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:02.077416 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" event={"ID":"8571bdb3-1773-450e-bd43-7f32bd7e52e0","Type":"ContainerStarted","Data":"67e124204d1577b1cd79d8c3908a128baee95e1d620229e1fc1e33af477cd3a4"} Jul 24 15:26:02.077878 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:02.077526 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:26:02.099183 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:02.099121 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" podStartSLOduration=1.637239984 podStartE2EDuration="6.099103974s" totalImagesPullingTime="3.409126113s" totalInitContainerRuntime="1.052737877s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:25:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:02.098759335 +0000 UTC m=+364.022684672" watchObservedRunningTime="2026-07-24 15:26:02.099103974 +0000 UTC m=+364.023029284" Jul 24 15:26:13.083470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:13.083433 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f445d9dcf-stggk" Jul 24 15:26:16.155874 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.155839 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9"] Jul 24 15:26:16.165985 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.165955 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.169859 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.169838 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 15:26:16.170106 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.170088 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vsxll\"" Jul 24 15:26:16.170204 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.170101 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 15:26:16.170204 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.170096 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 15:26:16.170204 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.170093 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 15:26:16.179503 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.179483 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9"] Jul 24 15:26:16.286140 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.286106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fwv5\" (UniqueName: \"kubernetes.io/projected/f69159b4-7150-4e9e-91d4-71ac49e5be4a-kube-api-access-4fwv5\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.286332 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.286171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f69159b4-7150-4e9e-91d4-71ac49e5be4a-tls-certs\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.286332 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.286238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f69159b4-7150-4e9e-91d4-71ac49e5be4a-tmp\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.311412 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.311369 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zpwsm"] Jul 24 15:26:16.314763 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.314741 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.318685 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.318639 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jv82x\"" Jul 24 15:26:16.318808 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.318637 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 24 15:26:16.333849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.333821 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zpwsm"] Jul 24 15:26:16.386658 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.386627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fwv5\" (UniqueName: \"kubernetes.io/projected/f69159b4-7150-4e9e-91d4-71ac49e5be4a-kube-api-access-4fwv5\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.386843 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.386690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctf9q\" (UniqueName: \"kubernetes.io/projected/101c1923-d051-4a02-bf28-99b66bc1029b-kube-api-access-ctf9q\") pod \"odh-model-controller-7998c67dc4-zpwsm\" (UID: \"101c1923-d051-4a02-bf28-99b66bc1029b\") " pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.386843 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.386731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/101c1923-d051-4a02-bf28-99b66bc1029b-cert\") pod \"odh-model-controller-7998c67dc4-zpwsm\" (UID: \"101c1923-d051-4a02-bf28-99b66bc1029b\") " pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.386843 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.386781 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f69159b4-7150-4e9e-91d4-71ac49e5be4a-tls-certs\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.386843 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.386829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f69159b4-7150-4e9e-91d4-71ac49e5be4a-tmp\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.389313 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.389284 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f69159b4-7150-4e9e-91d4-71ac49e5be4a-tmp\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.389517 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.389501 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f69159b4-7150-4e9e-91d4-71ac49e5be4a-tls-certs\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.396470 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.396444 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fwv5\" (UniqueName: \"kubernetes.io/projected/f69159b4-7150-4e9e-91d4-71ac49e5be4a-kube-api-access-4fwv5\") pod \"kube-auth-proxy-57b97974c4-lwhc9\" (UID: \"f69159b4-7150-4e9e-91d4-71ac49e5be4a\") " pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.475406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.475375 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" Jul 24 15:26:16.487804 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.487778 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctf9q\" (UniqueName: \"kubernetes.io/projected/101c1923-d051-4a02-bf28-99b66bc1029b-kube-api-access-ctf9q\") pod \"odh-model-controller-7998c67dc4-zpwsm\" (UID: \"101c1923-d051-4a02-bf28-99b66bc1029b\") " pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.487925 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.487821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/101c1923-d051-4a02-bf28-99b66bc1029b-cert\") pod \"odh-model-controller-7998c67dc4-zpwsm\" (UID: \"101c1923-d051-4a02-bf28-99b66bc1029b\") " pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.490959 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.490934 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/101c1923-d051-4a02-bf28-99b66bc1029b-cert\") pod \"odh-model-controller-7998c67dc4-zpwsm\" (UID: \"101c1923-d051-4a02-bf28-99b66bc1029b\") " pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.503969 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.503933 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctf9q\" (UniqueName: \"kubernetes.io/projected/101c1923-d051-4a02-bf28-99b66bc1029b-kube-api-access-ctf9q\") pod \"odh-model-controller-7998c67dc4-zpwsm\" (UID: \"101c1923-d051-4a02-bf28-99b66bc1029b\") " pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.626527 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.626482 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:16.639473 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.639446 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9"] Jul 24 15:26:16.642326 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:26:16.642279 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69159b4_7150_4e9e_91d4_71ac49e5be4a.slice/crio-942a1d71943b36370e9bcf25cb325705d1ba4dad0a93643d749f0ef9b980be6f WatchSource:0}: Error finding container 942a1d71943b36370e9bcf25cb325705d1ba4dad0a93643d749f0ef9b980be6f: Status 404 returned error can't find the container with id 942a1d71943b36370e9bcf25cb325705d1ba4dad0a93643d749f0ef9b980be6f Jul 24 15:26:16.774220 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:16.774124 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zpwsm"] Jul 24 15:26:16.776145 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:26:16.776120 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101c1923_d051_4a02_bf28_99b66bc1029b.slice/crio-f7f529478b8295a76cba8a22bdc630f4d29c768074ff0a2f7c48e4e92b3bb0a4 WatchSource:0}: Error finding container f7f529478b8295a76cba8a22bdc630f4d29c768074ff0a2f7c48e4e92b3bb0a4: Status 404 returned error can't find the container with id f7f529478b8295a76cba8a22bdc630f4d29c768074ff0a2f7c48e4e92b3bb0a4 Jul 24 15:26:17.133889 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.133797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" event={"ID":"101c1923-d051-4a02-bf28-99b66bc1029b","Type":"ContainerStarted","Data":"f7f529478b8295a76cba8a22bdc630f4d29c768074ff0a2f7c48e4e92b3bb0a4"} Jul 24 15:26:17.134925 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.134901 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" event={"ID":"f69159b4-7150-4e9e-91d4-71ac49e5be4a","Type":"ContainerStarted","Data":"942a1d71943b36370e9bcf25cb325705d1ba4dad0a93643d749f0ef9b980be6f"} Jul 24 15:26:17.244226 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.244187 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m5rpq"] Jul 24 15:26:17.247407 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.247383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.250489 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.250463 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 15:26:17.281632 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.281608 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m5rpq"] Jul 24 15:26:17.296421 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.296392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43851c5b-baa5-4c82-95cb-43c7fcb89dea-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.296566 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.296444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plrjk\" (UniqueName: \"kubernetes.io/projected/43851c5b-baa5-4c82-95cb-43c7fcb89dea-kube-api-access-plrjk\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.399995 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.399904 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43851c5b-baa5-4c82-95cb-43c7fcb89dea-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.399995 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.399967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plrjk\" (UniqueName: \"kubernetes.io/projected/43851c5b-baa5-4c82-95cb-43c7fcb89dea-kube-api-access-plrjk\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.400221 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:26:17.400070 2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 24 15:26:17.400221 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:26:17.400163 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43851c5b-baa5-4c82-95cb-43c7fcb89dea-tls-certs podName:43851c5b-baa5-4c82-95cb-43c7fcb89dea nodeName:}" failed. No retries permitted until 2026-07-24 15:26:17.900140492 +0000 UTC m=+379.824065785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/43851c5b-baa5-4c82-95cb-43c7fcb89dea-tls-certs") pod "model-serving-api-54d89bd5fc-m5rpq" (UID: "43851c5b-baa5-4c82-95cb-43c7fcb89dea") : secret "model-serving-api-tls" not found Jul 24 15:26:17.420593 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.420534 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plrjk\" (UniqueName: \"kubernetes.io/projected/43851c5b-baa5-4c82-95cb-43c7fcb89dea-kube-api-access-plrjk\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.544719 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.544669 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz"] Jul 24 15:26:17.549502 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.549468 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.552207 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.552175 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hlptv\"" Jul 24 15:26:17.552391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.552360 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 15:26:17.552391 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.552377 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 15:26:17.552729 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.552679 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 15:26:17.556001 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.555975 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz"] Jul 24 15:26:17.556203 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.556186 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 15:26:17.605397 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.605397 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605397 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.605682 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.605682 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wcqn\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-kube-api-access-9wcqn\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.605682 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbf55e-f56a-4729-b4ee-ef54c985934f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.605682 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.605906 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.605702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706443 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706669 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706456 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706669 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706669 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wcqn\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-kube-api-access-9wcqn\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706669 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706628 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbf55e-f56a-4729-b4ee-ef54c985934f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706905 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.706905 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.706705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.708479 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.708455 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.712772 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.712749 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.714438 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.714392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.715875 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.715853 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.717074 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.717051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wcqn\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-kube-api-access-9wcqn\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.717544 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.717467 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbf55e-f56a-4729-b4ee-ef54c985934f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.717748 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.717630 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8cghz\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.868290 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.868003 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:17.908980 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.908479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43851c5b-baa5-4c82-95cb-43c7fcb89dea-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:17.912571 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:17.912523 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43851c5b-baa5-4c82-95cb-43c7fcb89dea-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5rpq\" (UID: \"43851c5b-baa5-4c82-95cb-43c7fcb89dea\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:18.060365 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:18.060316 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz"] Jul 24 15:26:18.064193 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:26:18.064158 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedbf55e_f56a_4729_b4ee_ef54c985934f.slice/crio-28dceded20eb2532128b565ccd7763453e3da815232675c0961032b277d3cdcd WatchSource:0}: Error finding container 28dceded20eb2532128b565ccd7763453e3da815232675c0961032b277d3cdcd: Status 404 returned error can't find the container with id 28dceded20eb2532128b565ccd7763453e3da815232675c0961032b277d3cdcd Jul 24 15:26:18.144840 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:18.144799 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" event={"ID":"aedbf55e-f56a-4729-b4ee-ef54c985934f","Type":"ContainerStarted","Data":"28dceded20eb2532128b565ccd7763453e3da815232675c0961032b277d3cdcd"} Jul 24 15:26:18.158211 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:18.158182 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:18.365692 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:18.365594 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m5rpq"] Jul 24 15:26:18.369800 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:26:18.369769 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43851c5b_baa5_4c82_95cb_43c7fcb89dea.slice/crio-c5f5c40efbb7c07b0775f8e90de4e6e08cd83e3f11b1c03fe5620f4e85de739d WatchSource:0}: Error finding container c5f5c40efbb7c07b0775f8e90de4e6e08cd83e3f11b1c03fe5620f4e85de739d: Status 404 returned error can't find the container with id c5f5c40efbb7c07b0775f8e90de4e6e08cd83e3f11b1c03fe5620f4e85de739d Jul 24 15:26:19.159273 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:19.159210 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" event={"ID":"43851c5b-baa5-4c82-95cb-43c7fcb89dea","Type":"ContainerStarted","Data":"c5f5c40efbb7c07b0775f8e90de4e6e08cd83e3f11b1c03fe5620f4e85de739d"} Jul 24 15:26:21.173398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:21.173357 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" event={"ID":"101c1923-d051-4a02-bf28-99b66bc1029b","Type":"ContainerStarted","Data":"0676dfe5e2a96a291be5a1d747d7a9a637918c7a04b0a563f92a6767d6d362d3"} Jul 24 15:26:21.173823 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:21.173535 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:21.176140 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:21.176112 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" event={"ID":"f69159b4-7150-4e9e-91d4-71ac49e5be4a","Type":"ContainerStarted","Data":"0d351be22d0e992c07350d4e375dcc3ca0d0f4684ee6ff0305108ede629b370b"} Jul 24 15:26:21.193035 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:21.192275 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" podStartSLOduration=1.286881819 podStartE2EDuration="5.192255075s" totalImagesPullingTime="3.905373256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:21.190444508 +0000 UTC m=+383.114369818" watchObservedRunningTime="2026-07-24 15:26:21.192255075 +0000 UTC m=+383.116180386" Jul 24 15:26:21.209019 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:21.208946 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57b97974c4-lwhc9" podStartSLOduration=1.169576523 podStartE2EDuration="5.208924516s" totalImagesPullingTime="4.039347993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:21.207592172 +0000 UTC m=+383.131517491" watchObservedRunningTime="2026-07-24 15:26:21.208924516 +0000 UTC m=+383.132849827" Jul 24 15:26:22.183119 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:22.183080 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" event={"ID":"43851c5b-baa5-4c82-95cb-43c7fcb89dea","Type":"ContainerStarted","Data":"f47d6b4bc9a5c163fd3854fdbf30a5867ca823be90d57be54e132edac3a56889"} Jul 24 15:26:22.203886 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:22.203833 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" podStartSLOduration=2.066743408 podStartE2EDuration="5.203816531s" totalImagesPullingTime="3.137073123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:22.201270427 +0000 UTC m=+384.125195762" watchObservedRunningTime="2026-07-24 15:26:22.203816531 +0000 UTC m=+384.127741844" Jul 24 15:26:23.049773 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:23.049732 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:26:23.049882 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:23.049806 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:26:23.189623 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:23.189586 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" event={"ID":"aedbf55e-f56a-4729-b4ee-ef54c985934f","Type":"ContainerStarted","Data":"5565e69bed7a53d8f89ff40145b8aed768610bbd275267c9c9f9508121f0d468"} Jul 24 15:26:23.190119 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:23.189833 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:26:23.190119 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:23.189883 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:23.212165 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:23.212104 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" podStartSLOduration=1.2289954459999999 podStartE2EDuration="6.212083928s" totalImagesPullingTime="4.983088482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:23.20925763 +0000 UTC m=+385.133182940" watchObservedRunningTime="2026-07-24 15:26:23.212083928 +0000 UTC m=+385.136009236" Jul 24 15:26:24.197312 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:24.197274 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-8cghz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 15:26:24.197726 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:24.197336 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 15:26:27.196722 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:27.196686 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:28.027640 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.027606 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c97lt"] Jul 24 15:26:28.031392 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.031371 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg"] Jul 24 15:26:28.031527 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.031510 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.034543 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.034336 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 24 15:26:28.034684 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.034558 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4fmw9\"" Jul 24 15:26:28.034843 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.034826 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.037041 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.037022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-z5wtg\"" Jul 24 15:26:28.037142 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.037131 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 24 15:26:28.043402 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.043382 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c97lt"] Jul 24 15:26:28.046602 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.046549 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg"] Jul 24 15:26:28.205841 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.205795 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67bzc\" (UniqueName: \"kubernetes.io/projected/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-kube-api-access-67bzc\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.206230 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.205911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g64z\" (UniqueName: \"kubernetes.io/projected/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-kube-api-access-7g64z\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.206230 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.205974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.206230 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.206018 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-cert\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.306584 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.306497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67bzc\" (UniqueName: \"kubernetes.io/projected/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-kube-api-access-67bzc\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.306584 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.306565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g64z\" (UniqueName: \"kubernetes.io/projected/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-kube-api-access-7g64z\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.306846 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.306610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.306846 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.306668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-cert\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.306846 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:26:28.306739 2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 24 15:26:28.306846 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:26:28.306743 2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 24 15:26:28.306846 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:26:28.306814 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-cert podName:caacabef-2f1f-4aaa-91da-b0f8cdf1b322 nodeName:}" failed. No retries permitted until 2026-07-24 15:26:28.806792016 +0000 UTC m=+390.730717302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-cert") pod "llmisvc-controller-manager-777c6bd7d5-zmkdg" (UID: "caacabef-2f1f-4aaa-91da-b0f8cdf1b322") : secret "llmisvc-webhook-server-cert" not found Jul 24 15:26:28.306846 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:26:28.306835 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-cert podName:c2b03f75-deaf-4b23-9fa3-1bc5a362a003 nodeName:}" failed. No retries permitted until 2026-07-24 15:26:28.806825341 +0000 UTC m=+390.730750633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-cert") pod "kserve-controller-manager-d6f4b65b8-c97lt" (UID: "c2b03f75-deaf-4b23-9fa3-1bc5a362a003") : secret "kserve-webhook-server-cert" not found Jul 24 15:26:28.321766 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.321739 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67bzc\" (UniqueName: \"kubernetes.io/projected/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-kube-api-access-67bzc\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.322161 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.322138 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g64z\" (UniqueName: \"kubernetes.io/projected/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-kube-api-access-7g64z\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.811371 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.811335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.811568 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.811391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-cert\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.813897 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.813875 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caacabef-2f1f-4aaa-91da-b0f8cdf1b322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zmkdg\" (UID: \"caacabef-2f1f-4aaa-91da-b0f8cdf1b322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:28.813897 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.813895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2b03f75-deaf-4b23-9fa3-1bc5a362a003-cert\") pod \"kserve-controller-manager-d6f4b65b8-c97lt\" (UID: \"c2b03f75-deaf-4b23-9fa3-1bc5a362a003\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.947122 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.947093 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:28.954171 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:28.954142 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:29.088963 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:29.088350 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c97lt"] Jul 24 15:26:29.102007 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:26:29.101765 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b03f75_deaf_4b23_9fa3_1bc5a362a003.slice/crio-121a22f9819474234ef0c8b918221e4cdf64212c3db108a824f8c89725e8fd67 WatchSource:0}: Error finding container 121a22f9819474234ef0c8b918221e4cdf64212c3db108a824f8c89725e8fd67: Status 404 returned error can't find the container with id 121a22f9819474234ef0c8b918221e4cdf64212c3db108a824f8c89725e8fd67 Jul 24 15:26:29.127227 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:29.127203 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg"] Jul 24 15:26:29.130066 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:26:29.130043 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcaacabef_2f1f_4aaa_91da_b0f8cdf1b322.slice/crio-31eb58ee41271f54c2aacee660c4367dae91f13efe6a3b5eee59b20f3e748183 WatchSource:0}: Error finding container 31eb58ee41271f54c2aacee660c4367dae91f13efe6a3b5eee59b20f3e748183: Status 404 returned error can't find the container with id 31eb58ee41271f54c2aacee660c4367dae91f13efe6a3b5eee59b20f3e748183 Jul 24 15:26:29.220300 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:29.220262 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" event={"ID":"caacabef-2f1f-4aaa-91da-b0f8cdf1b322","Type":"ContainerStarted","Data":"31eb58ee41271f54c2aacee660c4367dae91f13efe6a3b5eee59b20f3e748183"} Jul 24 15:26:29.221348 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:29.221323 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" event={"ID":"c2b03f75-deaf-4b23-9fa3-1bc5a362a003","Type":"ContainerStarted","Data":"121a22f9819474234ef0c8b918221e4cdf64212c3db108a824f8c89725e8fd67"} Jul 24 15:26:30.062606 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.062569 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz"] Jul 24 15:26:30.062890 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.062843 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerName="discovery" containerID="cri-o://5565e69bed7a53d8f89ff40145b8aed768610bbd275267c9c9f9508121f0d468" gracePeriod=30 Jul 24 15:26:30.195538 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.195497 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-8cghz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.22:8080/ready\": dial tcp 10.134.0.22:8080: connect: connection refused" start-of-body= Jul 24 15:26:30.195607 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.195556 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.22:8080/ready\": dial tcp 10.134.0.22:8080: connect: connection refused" Jul 24 15:26:30.229053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.228936 2564 generic.go:356] "Generic (PLEG): container finished" podID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerID="5565e69bed7a53d8f89ff40145b8aed768610bbd275267c9c9f9508121f0d468" exitCode=0 Jul 24 15:26:30.229053 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.228988 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" event={"ID":"aedbf55e-f56a-4729-b4ee-ef54c985934f","Type":"ContainerDied","Data":"5565e69bed7a53d8f89ff40145b8aed768610bbd275267c9c9f9508121f0d468"} Jul 24 15:26:30.405011 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.404985 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:30.528891 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.528858 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-dns-cert\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529056 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.528921 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-ca-configmap\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529056 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.528967 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-cacerts\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529056 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.529051 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-kubeconfig\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529245 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.529076 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbf55e-f56a-4729-b4ee-ef54c985934f-local-certs\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529245 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.529109 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-token\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529245 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.529134 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wcqn\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-kube-api-access-9wcqn\") pod \"aedbf55e-f56a-4729-b4ee-ef54c985934f\" (UID: \"aedbf55e-f56a-4729-b4ee-ef54c985934f\") " Jul 24 15:26:30.529458 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.529358 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:26:30.532058 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.532028 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-kube-api-access-9wcqn" (OuterVolumeSpecName: "kube-api-access-9wcqn") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "kube-api-access-9wcqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:26:30.532190 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.532027 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-token" (OuterVolumeSpecName: "istio-token") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:26:30.532769 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.532701 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:26:30.532769 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.532723 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-cacerts" (OuterVolumeSpecName: "cacerts") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:26:30.534305 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.534275 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:26:30.535487 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.535459 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aedbf55e-f56a-4729-b4ee-ef54c985934f-local-certs" (OuterVolumeSpecName: "local-certs") pod "aedbf55e-f56a-4729-b4ee-ef54c985934f" (UID: "aedbf55e-f56a-4729-b4ee-ef54c985934f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:26:30.630941 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.630909 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-kubeconfig\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:30.630941 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.630942 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbf55e-f56a-4729-b4ee-ef54c985934f-local-certs\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:30.630941 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.630957 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-token\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:30.631246 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.630970 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wcqn\" (UniqueName: \"kubernetes.io/projected/aedbf55e-f56a-4729-b4ee-ef54c985934f-kube-api-access-9wcqn\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:30.631246 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.630986 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-dns-cert\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:30.631246 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.631000 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aedbf55e-f56a-4729-b4ee-ef54c985934f-istio-csr-ca-configmap\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:30.631246 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:30.631012 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aedbf55e-f56a-4729-b4ee-ef54c985934f-cacerts\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:26:31.236668 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:31.236613 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" event={"ID":"aedbf55e-f56a-4729-b4ee-ef54c985934f","Type":"ContainerDied","Data":"28dceded20eb2532128b565ccd7763453e3da815232675c0961032b277d3cdcd"} Jul 24 15:26:31.237100 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:31.236708 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz" Jul 24 15:26:31.237100 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:31.236714 2564 scope.go:117] "RemoveContainer" containerID="5565e69bed7a53d8f89ff40145b8aed768610bbd275267c9c9f9508121f0d468" Jul 24 15:26:31.257124 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:31.257093 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz"] Jul 24 15:26:31.262299 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:31.262272 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8cghz"] Jul 24 15:26:32.189137 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:32.189102 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-zpwsm" Jul 24 15:26:32.534529 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:32.534454 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" path="/var/lib/kubelet/pods/aedbf55e-f56a-4729-b4ee-ef54c985934f/volumes" Jul 24 15:26:33.248617 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:33.248583 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" event={"ID":"c2b03f75-deaf-4b23-9fa3-1bc5a362a003","Type":"ContainerStarted","Data":"889b38407f82dd3aef59fc04ee54594968ad83294326ec32bdf6200fa6b5f254"} Jul 24 15:26:33.248839 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:33.248744 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:26:33.249982 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:33.249961 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" event={"ID":"caacabef-2f1f-4aaa-91da-b0f8cdf1b322","Type":"ContainerStarted","Data":"fd5cd585590166633b9964cdc07ebf4b8fbd6bfbead83499353f3221ab5a6aba"} Jul 24 15:26:33.250099 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:33.250085 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:26:33.271817 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:33.271767 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" podStartSLOduration=3.141487671 podStartE2EDuration="6.271751588s" totalImagesPullingTime="3.130263917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:33.270035655 +0000 UTC m=+395.193960991" watchObservedRunningTime="2026-07-24 15:26:33.271751588 +0000 UTC m=+395.195676930" Jul 24 15:26:33.288522 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:33.288460 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" podStartSLOduration=3.184542768 podStartE2EDuration="6.288442045s" totalImagesPullingTime="3.103899277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:26:33.287241272 +0000 UTC m=+395.211166608" watchObservedRunningTime="2026-07-24 15:26:33.288442045 +0000 UTC m=+395.212367354" Jul 24 15:26:34.199449 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:26:34.199416 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-m5rpq" Jul 24 15:27:04.257113 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:04.257083 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zmkdg" Jul 24 15:27:04.259977 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:04.259958 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c97lt" Jul 24 15:27:17.267485 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.267447 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j78mk"] Jul 24 15:27:17.268086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.267775 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerName="discovery" Jul 24 15:27:17.268086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.267786 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerName="discovery" Jul 24 15:27:17.268086 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.267842 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aedbf55e-f56a-4729-b4ee-ef54c985934f" containerName="discovery" Jul 24 15:27:17.277800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.277772 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:17.281193 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.281164 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j78mk"] Jul 24 15:27:17.282579 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.282554 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mjxzs\"" Jul 24 15:27:17.282767 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.282748 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 15:27:17.282840 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.282826 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 15:27:17.448582 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.448554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6tqt\" (UniqueName: \"kubernetes.io/projected/f83f60ba-6c25-4c24-a748-a872f4497bd8-kube-api-access-g6tqt\") pod \"kuadrant-operator-catalog-j78mk\" (UID: \"f83f60ba-6c25-4c24-a748-a872f4497bd8\") " pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:17.550057 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.549983 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6tqt\" (UniqueName: \"kubernetes.io/projected/f83f60ba-6c25-4c24-a748-a872f4497bd8-kube-api-access-g6tqt\") pod \"kuadrant-operator-catalog-j78mk\" (UID: \"f83f60ba-6c25-4c24-a748-a872f4497bd8\") " pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:17.563471 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.563441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6tqt\" (UniqueName: \"kubernetes.io/projected/f83f60ba-6c25-4c24-a748-a872f4497bd8-kube-api-access-g6tqt\") pod \"kuadrant-operator-catalog-j78mk\" (UID: \"f83f60ba-6c25-4c24-a748-a872f4497bd8\") " pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:17.591780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.591755 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:17.640744 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.640711 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j78mk"] Jul 24 15:27:17.725881 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.725808 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j78mk"] Jul 24 15:27:17.729431 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:27:17.729393 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83f60ba_6c25_4c24_a748_a872f4497bd8.slice/crio-c0e2621eda92d1d4c154ddbf24fff6329febf306b6bb2955f2c5e41da8ed1848 WatchSource:0}: Error finding container c0e2621eda92d1d4c154ddbf24fff6329febf306b6bb2955f2c5e41da8ed1848: Status 404 returned error can't find the container with id c0e2621eda92d1d4c154ddbf24fff6329febf306b6bb2955f2c5e41da8ed1848 Jul 24 15:27:17.847076 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.846997 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sjnkn"] Jul 24 15:27:17.849990 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.849974 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:17.857697 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.857675 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sjnkn"] Jul 24 15:27:17.953885 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:17.953848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4lrt\" (UniqueName: \"kubernetes.io/projected/37c57504-b8be-4d0b-8f79-c35c358c284a-kube-api-access-l4lrt\") pod \"kuadrant-operator-catalog-sjnkn\" (UID: \"37c57504-b8be-4d0b-8f79-c35c358c284a\") " pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:18.054738 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:18.054674 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4lrt\" (UniqueName: \"kubernetes.io/projected/37c57504-b8be-4d0b-8f79-c35c358c284a-kube-api-access-l4lrt\") pod \"kuadrant-operator-catalog-sjnkn\" (UID: \"37c57504-b8be-4d0b-8f79-c35c358c284a\") " pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:18.064035 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:18.064011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4lrt\" (UniqueName: \"kubernetes.io/projected/37c57504-b8be-4d0b-8f79-c35c358c284a-kube-api-access-l4lrt\") pod \"kuadrant-operator-catalog-sjnkn\" (UID: \"37c57504-b8be-4d0b-8f79-c35c358c284a\") " pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:18.159570 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:18.159478 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:18.303508 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:18.303372 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sjnkn"] Jul 24 15:27:18.322664 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:27:18.322616 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c57504_b8be_4d0b_8f79_c35c358c284a.slice/crio-409d7684aa163c891fb0a468d624e3122af94c5aff38c352b896ea93e70a43ec WatchSource:0}: Error finding container 409d7684aa163c891fb0a468d624e3122af94c5aff38c352b896ea93e70a43ec: Status 404 returned error can't find the container with id 409d7684aa163c891fb0a468d624e3122af94c5aff38c352b896ea93e70a43ec Jul 24 15:27:18.442187 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:18.442151 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" event={"ID":"f83f60ba-6c25-4c24-a748-a872f4497bd8","Type":"ContainerStarted","Data":"c0e2621eda92d1d4c154ddbf24fff6329febf306b6bb2955f2c5e41da8ed1848"} Jul 24 15:27:18.443615 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:18.443580 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" event={"ID":"37c57504-b8be-4d0b-8f79-c35c358c284a","Type":"ContainerStarted","Data":"409d7684aa163c891fb0a468d624e3122af94c5aff38c352b896ea93e70a43ec"} Jul 24 15:27:20.454984 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.454944 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" event={"ID":"37c57504-b8be-4d0b-8f79-c35c358c284a","Type":"ContainerStarted","Data":"9b9cfa213c1bca85dcfb05bdafac5942811ab734f33ba5a320261f7c7ac07a41"} Jul 24 15:27:20.456368 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.456344 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" event={"ID":"f83f60ba-6c25-4c24-a748-a872f4497bd8","Type":"ContainerStarted","Data":"1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a"} Jul 24 15:27:20.456473 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.456420 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" podUID="f83f60ba-6c25-4c24-a748-a872f4497bd8" containerName="registry-server" containerID="cri-o://1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a" gracePeriod=2 Jul 24 15:27:20.472820 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.472776 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" podStartSLOduration=2.106556884 podStartE2EDuration="3.472761292s" totalImagesPullingTime="1.366204408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:27:20.470617668 +0000 UTC m=+442.394542976" watchObservedRunningTime="2026-07-24 15:27:20.472761292 +0000 UTC m=+442.396686601" Jul 24 15:27:20.487565 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.487521 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" podStartSLOduration=1.5313562090000001 podStartE2EDuration="3.487508845s" totalImagesPullingTime="1.956152636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:27:20.485890317 +0000 UTC m=+442.409815627" watchObservedRunningTime="2026-07-24 15:27:20.487508845 +0000 UTC m=+442.411434152" Jul 24 15:27:20.698576 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.698556 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:20.773800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.773719 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6tqt\" (UniqueName: \"kubernetes.io/projected/f83f60ba-6c25-4c24-a748-a872f4497bd8-kube-api-access-g6tqt\") pod \"f83f60ba-6c25-4c24-a748-a872f4497bd8\" (UID: \"f83f60ba-6c25-4c24-a748-a872f4497bd8\") " Jul 24 15:27:20.776021 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.775991 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83f60ba-6c25-4c24-a748-a872f4497bd8-kube-api-access-g6tqt" (OuterVolumeSpecName: "kube-api-access-g6tqt") pod "f83f60ba-6c25-4c24-a748-a872f4497bd8" (UID: "f83f60ba-6c25-4c24-a748-a872f4497bd8"). InnerVolumeSpecName "kube-api-access-g6tqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:27:20.874210 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:20.874185 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6tqt\" (UniqueName: \"kubernetes.io/projected/f83f60ba-6c25-4c24-a748-a872f4497bd8-kube-api-access-g6tqt\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:27:21.461782 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.461751 2564 generic.go:356] "Generic (PLEG): container finished" podID="f83f60ba-6c25-4c24-a748-a872f4497bd8" containerID="1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a" exitCode=0 Jul 24 15:27:21.462202 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.461790 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" Jul 24 15:27:21.462202 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.461832 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" event={"ID":"f83f60ba-6c25-4c24-a748-a872f4497bd8","Type":"ContainerDied","Data":"1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a"} Jul 24 15:27:21.462202 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.461875 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j78mk" event={"ID":"f83f60ba-6c25-4c24-a748-a872f4497bd8","Type":"ContainerDied","Data":"c0e2621eda92d1d4c154ddbf24fff6329febf306b6bb2955f2c5e41da8ed1848"} Jul 24 15:27:21.462202 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.461894 2564 scope.go:117] "RemoveContainer" containerID="1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a" Jul 24 15:27:21.471430 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.471415 2564 scope.go:117] "RemoveContainer" containerID="1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a" Jul 24 15:27:21.471699 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:27:21.471682 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a\": container with ID starting with 1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a not found: ID does not exist" containerID="1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a" Jul 24 15:27:21.471750 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.471709 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a"} err="failed to get container status \"1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a\": rpc error: code = NotFound desc = could not find container \"1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a\": container with ID starting with 1e1966e305f6827db6c514c8eb4c5e9a9a6c84ad7b8309ce0202943ff1780e5a not found: ID does not exist" Jul 24 15:27:21.483779 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.483754 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j78mk"] Jul 24 15:27:21.489812 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:21.489793 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j78mk"] Jul 24 15:27:22.533680 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:22.533632 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83f60ba-6c25-4c24-a748-a872f4497bd8" path="/var/lib/kubelet/pods/f83f60ba-6c25-4c24-a748-a872f4497bd8/volumes" Jul 24 15:27:28.160753 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:28.160716 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:28.161358 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:28.160769 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:28.182260 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:28.182232 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:28.512946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:28.512919 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-sjnkn" Jul 24 15:27:49.677532 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.677488 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh"] Jul 24 15:27:49.678122 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.677980 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83f60ba-6c25-4c24-a748-a872f4497bd8" containerName="registry-server" Jul 24 15:27:49.678122 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.678001 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83f60ba-6c25-4c24-a748-a872f4497bd8" containerName="registry-server" Jul 24 15:27:49.678122 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.678107 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83f60ba-6c25-4c24-a748-a872f4497bd8" containerName="registry-server" Jul 24 15:27:49.680043 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.680020 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:27:49.684489 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.684458 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 24 15:27:49.684606 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.684506 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zzj7p\"" Jul 24 15:27:49.693722 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.693698 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh"] Jul 24 15:27:49.808947 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.808915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r76p\" (UniqueName: \"kubernetes.io/projected/5a205938-52a1-47ba-bfca-0703c77fb2fa-kube-api-access-4r76p\") pod \"dns-operator-controller-manager-864dd78c4c-g2dnh\" (UID: \"5a205938-52a1-47ba-bfca-0703c77fb2fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:27:49.910249 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.910210 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r76p\" (UniqueName: \"kubernetes.io/projected/5a205938-52a1-47ba-bfca-0703c77fb2fa-kube-api-access-4r76p\") pod \"dns-operator-controller-manager-864dd78c4c-g2dnh\" (UID: \"5a205938-52a1-47ba-bfca-0703c77fb2fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:27:49.927864 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.927798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r76p\" (UniqueName: \"kubernetes.io/projected/5a205938-52a1-47ba-bfca-0703c77fb2fa-kube-api-access-4r76p\") pod \"dns-operator-controller-manager-864dd78c4c-g2dnh\" (UID: \"5a205938-52a1-47ba-bfca-0703c77fb2fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:27:49.990314 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:49.990278 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:27:50.133688 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:50.133641 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh"] Jul 24 15:27:50.136683 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:27:50.136631 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a205938_52a1_47ba_bfca_0703c77fb2fa.slice/crio-91e8c09475fb9d64ebd3e1a1b375a44e226665626450e21d06f3e3b4d9bdf374 WatchSource:0}: Error finding container 91e8c09475fb9d64ebd3e1a1b375a44e226665626450e21d06f3e3b4d9bdf374: Status 404 returned error can't find the container with id 91e8c09475fb9d64ebd3e1a1b375a44e226665626450e21d06f3e3b4d9bdf374 Jul 24 15:27:50.586663 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:50.586617 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" event={"ID":"5a205938-52a1-47ba-bfca-0703c77fb2fa","Type":"ContainerStarted","Data":"91e8c09475fb9d64ebd3e1a1b375a44e226665626450e21d06f3e3b4d9bdf374"} Jul 24 15:27:52.597266 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:52.597216 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" event={"ID":"5a205938-52a1-47ba-bfca-0703c77fb2fa","Type":"ContainerStarted","Data":"9b895da12e366628e7501ad7c2a9b611c81d04f253629a644c8cf5906628e3cb"} Jul 24 15:27:52.597723 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:52.597326 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:27:52.623459 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:52.623410 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" podStartSLOduration=1.926332857 podStartE2EDuration="3.623394164s" totalImagesPullingTime="1.697061307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:27:52.621957422 +0000 UTC m=+474.545882755" watchObservedRunningTime="2026-07-24 15:27:52.623394164 +0000 UTC m=+474.547319472" Jul 24 15:27:53.108231 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.108192 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg"] Jul 24 15:27:53.110699 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.110684 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:27:53.113500 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.113478 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g9jkl\"" Jul 24 15:27:53.121692 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.121671 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg"] Jul 24 15:27:53.241490 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.241453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dsz7\" (UniqueName: \"kubernetes.io/projected/f39be795-0da5-4121-9fac-48b3a565fed6-kube-api-access-9dsz7\") pod \"limitador-operator-controller-manager-859d7b78bd-frqbg\" (UID: \"f39be795-0da5-4121-9fac-48b3a565fed6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:27:53.342163 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.342121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dsz7\" (UniqueName: \"kubernetes.io/projected/f39be795-0da5-4121-9fac-48b3a565fed6-kube-api-access-9dsz7\") pod \"limitador-operator-controller-manager-859d7b78bd-frqbg\" (UID: \"f39be795-0da5-4121-9fac-48b3a565fed6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:27:53.354016 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.353984 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dsz7\" (UniqueName: \"kubernetes.io/projected/f39be795-0da5-4121-9fac-48b3a565fed6-kube-api-access-9dsz7\") pod \"limitador-operator-controller-manager-859d7b78bd-frqbg\" (UID: \"f39be795-0da5-4121-9fac-48b3a565fed6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:27:53.420188 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.420105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:27:53.556780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.556757 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg"] Jul 24 15:27:53.558671 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:27:53.558628 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39be795_0da5_4121_9fac_48b3a565fed6.slice/crio-2d59c97131cf3097b4f2525019f8be3138279d59f737b2e6027ca2e1dedf22e8 WatchSource:0}: Error finding container 2d59c97131cf3097b4f2525019f8be3138279d59f737b2e6027ca2e1dedf22e8: Status 404 returned error can't find the container with id 2d59c97131cf3097b4f2525019f8be3138279d59f737b2e6027ca2e1dedf22e8 Jul 24 15:27:53.602800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:53.602768 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" event={"ID":"f39be795-0da5-4121-9fac-48b3a565fed6","Type":"ContainerStarted","Data":"2d59c97131cf3097b4f2525019f8be3138279d59f737b2e6027ca2e1dedf22e8"} Jul 24 15:27:55.614141 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:55.614105 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" event={"ID":"f39be795-0da5-4121-9fac-48b3a565fed6","Type":"ContainerStarted","Data":"4aa013466e8217f4061f89cf13e70c42d572e31925751a7bc4571e9403b78843"} Jul 24 15:27:55.614633 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:55.614211 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:27:55.637692 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:55.637624 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" podStartSLOduration=1.36322121 podStartE2EDuration="2.637609845s" totalImagesPullingTime="1.274388635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:27:55.636980244 +0000 UTC m=+477.560905569" watchObservedRunningTime="2026-07-24 15:27:55.637609845 +0000 UTC m=+477.561535154" Jul 24 15:27:57.130405 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.130370 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-742nh"] Jul 24 15:27:57.133336 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.133316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:27:57.136133 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.136108 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cc9zr\"" Jul 24 15:27:57.148405 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.148384 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-742nh"] Jul 24 15:27:57.280482 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.280445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95pb8\" (UniqueName: \"kubernetes.io/projected/f38d2018-77f6-47e4-a25c-dd14651f00ed-kube-api-access-95pb8\") pod \"authorino-operator-75cc8d5cb7-742nh\" (UID: \"f38d2018-77f6-47e4-a25c-dd14651f00ed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:27:57.381787 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.381707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95pb8\" (UniqueName: \"kubernetes.io/projected/f38d2018-77f6-47e4-a25c-dd14651f00ed-kube-api-access-95pb8\") pod \"authorino-operator-75cc8d5cb7-742nh\" (UID: \"f38d2018-77f6-47e4-a25c-dd14651f00ed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:27:57.393919 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.393897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95pb8\" (UniqueName: \"kubernetes.io/projected/f38d2018-77f6-47e4-a25c-dd14651f00ed-kube-api-access-95pb8\") pod \"authorino-operator-75cc8d5cb7-742nh\" (UID: \"f38d2018-77f6-47e4-a25c-dd14651f00ed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:27:57.443309 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.443282 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:27:57.582261 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.582231 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-742nh"] Jul 24 15:27:57.584256 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:27:57.584228 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38d2018_77f6_47e4_a25c_dd14651f00ed.slice/crio-e5b379bd6ec80605910b74e8ab3e7490e5e574dfdd590ddcd4fba02e25bea70d WatchSource:0}: Error finding container e5b379bd6ec80605910b74e8ab3e7490e5e574dfdd590ddcd4fba02e25bea70d: Status 404 returned error can't find the container with id e5b379bd6ec80605910b74e8ab3e7490e5e574dfdd590ddcd4fba02e25bea70d Jul 24 15:27:57.623539 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:27:57.623503 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" event={"ID":"f38d2018-77f6-47e4-a25c-dd14651f00ed","Type":"ContainerStarted","Data":"e5b379bd6ec80605910b74e8ab3e7490e5e574dfdd590ddcd4fba02e25bea70d"} Jul 24 15:28:00.640553 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:00.640514 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" event={"ID":"f38d2018-77f6-47e4-a25c-dd14651f00ed","Type":"ContainerStarted","Data":"f0df9d4f6076c5af5ac28ec4bc3bd9b6b901be91fe6d786e6b29f02cda8da3cd"} Jul 24 15:28:00.640954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:00.640585 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:28:00.667717 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:00.667638 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" podStartSLOduration=1.333446578 podStartE2EDuration="3.667619098s" totalImagesPullingTime="2.33417252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:00.666797606 +0000 UTC m=+482.590722914" watchObservedRunningTime="2026-07-24 15:28:00.667619098 +0000 UTC m=+482.591544408" Jul 24 15:28:01.971579 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:01.971546 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc"] Jul 24 15:28:01.976954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:01.976919 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:01.980048 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:01.980024 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2f7nj\"" Jul 24 15:28:01.987256 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:01.987229 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc"] Jul 24 15:28:02.126744 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.126712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c90d8042-e7f0-4dc2-97fb-a634a6740206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.126934 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.126840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dk79\" (UniqueName: \"kubernetes.io/projected/c90d8042-e7f0-4dc2-97fb-a634a6740206-kube-api-access-8dk79\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.228189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.228096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c90d8042-e7f0-4dc2-97fb-a634a6740206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.228189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.228159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dk79\" (UniqueName: \"kubernetes.io/projected/c90d8042-e7f0-4dc2-97fb-a634a6740206-kube-api-access-8dk79\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.228520 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.228500 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c90d8042-e7f0-4dc2-97fb-a634a6740206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.239032 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.239008 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dk79\" (UniqueName: \"kubernetes.io/projected/c90d8042-e7f0-4dc2-97fb-a634a6740206-kube-api-access-8dk79\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.289548 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.289517 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:02.438143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.438122 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc"] Jul 24 15:28:02.440399 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:02.440370 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90d8042_e7f0_4dc2_97fb_a634a6740206.slice/crio-9587891e877ec2f0f71dacc9a95c4620c88fd0d91cecec66412a7a00a84e6f8c WatchSource:0}: Error finding container 9587891e877ec2f0f71dacc9a95c4620c88fd0d91cecec66412a7a00a84e6f8c: Status 404 returned error can't find the container with id 9587891e877ec2f0f71dacc9a95c4620c88fd0d91cecec66412a7a00a84e6f8c Jul 24 15:28:02.650522 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:02.650439 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" event={"ID":"c90d8042-e7f0-4dc2-97fb-a634a6740206","Type":"ContainerStarted","Data":"9587891e877ec2f0f71dacc9a95c4620c88fd0d91cecec66412a7a00a84e6f8c"} Jul 24 15:28:03.605974 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:03.605940 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g2dnh" Jul 24 15:28:06.621570 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:06.621542 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:07.677597 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:07.677559 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" event={"ID":"c90d8042-e7f0-4dc2-97fb-a634a6740206","Type":"ContainerStarted","Data":"014dda151b02d2cd2d0610e3fc839fe1131bf1ee2cd7aded51e73a4992b1f145"} Jul 24 15:28:07.678048 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:07.677712 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:07.702596 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:07.702545 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" podStartSLOduration=2.2904910640000002 podStartE2EDuration="6.702530671s" totalImagesPullingTime="4.412039607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:07.700304505 +0000 UTC m=+489.624229843" watchObservedRunningTime="2026-07-24 15:28:07.702530671 +0000 UTC m=+489.626455979" Jul 24 15:28:11.647751 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:11.647725 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-742nh" Jul 24 15:28:18.684553 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:18.684521 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:19.828813 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.828783 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn"] Jul 24 15:28:19.832513 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.832491 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:19.843939 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.843916 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn"] Jul 24 15:28:19.882314 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.882280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2bd7af7-474a-4b3c-9419-03ecf9691058-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:19.882481 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.882337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2744\" (UniqueName: \"kubernetes.io/projected/a2bd7af7-474a-4b3c-9419-03ecf9691058-kube-api-access-w2744\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:19.983045 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.983015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2bd7af7-474a-4b3c-9419-03ecf9691058-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:19.983221 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.983056 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2744\" (UniqueName: \"kubernetes.io/projected/a2bd7af7-474a-4b3c-9419-03ecf9691058-kube-api-access-w2744\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:19.983385 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.983365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2bd7af7-474a-4b3c-9419-03ecf9691058-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:19.992523 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:19.992499 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2744\" (UniqueName: \"kubernetes.io/projected/a2bd7af7-474a-4b3c-9419-03ecf9691058-kube-api-access-w2744\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:20.142858 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.142766 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:20.289493 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.289460 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn"] Jul 24 15:28:20.293507 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:20.293479 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2bd7af7_474a_4b3c_9419_03ecf9691058.slice/crio-95573dd347a9f91a2f543edc9112e89b18fd01d31578818cc50f56641340da20 WatchSource:0}: Error finding container 95573dd347a9f91a2f543edc9112e89b18fd01d31578818cc50f56641340da20: Status 404 returned error can't find the container with id 95573dd347a9f91a2f543edc9112e89b18fd01d31578818cc50f56641340da20 Jul 24 15:28:20.507862 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.507828 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn"] Jul 24 15:28:20.512769 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.512742 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn"] Jul 24 15:28:20.527060 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.527030 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc"] Jul 24 15:28:20.527327 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.527282 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" containerName="manager" containerID="cri-o://014dda151b02d2cd2d0610e3fc839fe1131bf1ee2cd7aded51e73a4992b1f145" gracePeriod=2 Jul 24 15:28:20.535439 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.535410 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x"] Jul 24 15:28:20.535861 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.535845 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" containerName="manager" Jul 24 15:28:20.535967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.535863 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" containerName="manager" Jul 24 15:28:20.535967 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.535958 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" containerName="manager" Jul 24 15:28:20.538957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.538936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.543396 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.543370 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc"] Jul 24 15:28:20.545518 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.545477 2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c90d8042-e7f0-4dc2-97fb-a634a6740206\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.549308 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.549258 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x"] Jul 24 15:28:20.552191 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.552168 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx"] Jul 24 15:28:20.552508 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.552496 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" containerName="manager" Jul 24 15:28:20.552561 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.552509 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" containerName="manager" Jul 24 15:28:20.552596 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.552564 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" containerName="manager" Jul 24 15:28:20.555398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.555374 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.556278 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.556262 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg"] Jul 24 15:28:20.556419 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.556402 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" containerName="manager" containerID="cri-o://4aa013466e8217f4061f89cf13e70c42d572e31925751a7bc4571e9403b78843" gracePeriod=2 Jul 24 15:28:20.571304 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.570925 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx"] Jul 24 15:28:20.573729 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.573670 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.573729 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.573705 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg"] Jul 24 15:28:20.575810 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.575778 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:20.577888 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.577865 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.581587 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.581567 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh"] Jul 24 15:28:20.581923 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.581909 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" containerName="manager" Jul 24 15:28:20.581923 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.581923 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" containerName="manager" Jul 24 15:28:20.582054 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.581989 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" containerName="manager" Jul 24 15:28:20.584782 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.584765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:20.589114 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.589090 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn6ld\" (UniqueName: \"kubernetes.io/projected/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-kube-api-access-sn6ld\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rz87x\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.589283 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.589203 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rz87x\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.589387 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.589297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p69gg\" (UniqueName: \"kubernetes.io/projected/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-kube-api-access-p69gg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-46jxx\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.589387 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.589343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-46jxx\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.595904 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.595875 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh"] Jul 24 15:28:20.609160 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.609124 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:20.610928 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.610902 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.690295 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgfrt\" (UniqueName: \"kubernetes.io/projected/0654f06f-5299-46a5-838b-ccbc00756c9a-kube-api-access-qgfrt\") pod \"limitador-operator-controller-manager-859d7b78bd-6nchh\" (UID: \"0654f06f-5299-46a5-838b-ccbc00756c9a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:20.690503 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rz87x\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.690503 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p69gg\" (UniqueName: \"kubernetes.io/projected/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-kube-api-access-p69gg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-46jxx\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.690503 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-46jxx\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.690503 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690437 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn6ld\" (UniqueName: \"kubernetes.io/projected/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-kube-api-access-sn6ld\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rz87x\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.690774 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690726 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rz87x\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.690879 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.690854 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-46jxx\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.706012 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.705961 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn6ld\" (UniqueName: \"kubernetes.io/projected/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-kube-api-access-sn6ld\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rz87x\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.706146 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.706109 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p69gg\" (UniqueName: \"kubernetes.io/projected/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-kube-api-access-p69gg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-46jxx\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.741773 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.741727 2564 generic.go:356] "Generic (PLEG): container finished" podID="c90d8042-e7f0-4dc2-97fb-a634a6740206" containerID="014dda151b02d2cd2d0610e3fc839fe1131bf1ee2cd7aded51e73a4992b1f145" exitCode=0 Jul 24 15:28:20.748153 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.748121 2564 generic.go:356] "Generic (PLEG): container finished" podID="f39be795-0da5-4121-9fac-48b3a565fed6" containerID="4aa013466e8217f4061f89cf13e70c42d572e31925751a7bc4571e9403b78843" exitCode=0 Jul 24 15:28:20.752717 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.752687 2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" container="manager" Jul 24 15:28:20.755257 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.755220 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:20.757368 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.757334 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.790901 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.790872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgfrt\" (UniqueName: \"kubernetes.io/projected/0654f06f-5299-46a5-838b-ccbc00756c9a-kube-api-access-qgfrt\") pod \"limitador-operator-controller-manager-859d7b78bd-6nchh\" (UID: \"0654f06f-5299-46a5-838b-ccbc00756c9a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:20.794227 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.794204 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.796268 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.796243 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:20.798090 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.798067 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.798674 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.798637 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:20.800480 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.800461 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:20.802406 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:20.802385 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:20.808268 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.808250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgfrt\" (UniqueName: \"kubernetes.io/projected/0654f06f-5299-46a5-838b-ccbc00756c9a-kube-api-access-qgfrt\") pod \"limitador-operator-controller-manager-859d7b78bd-6nchh\" (UID: \"0654f06f-5299-46a5-838b-ccbc00756c9a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:20.891387 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.891355 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c90d8042-e7f0-4dc2-97fb-a634a6740206-extensions-socket-volume\") pod \"c90d8042-e7f0-4dc2-97fb-a634a6740206\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " Jul 24 15:28:20.891827 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.891395 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dsz7\" (UniqueName: \"kubernetes.io/projected/f39be795-0da5-4121-9fac-48b3a565fed6-kube-api-access-9dsz7\") pod \"f39be795-0da5-4121-9fac-48b3a565fed6\" (UID: \"f39be795-0da5-4121-9fac-48b3a565fed6\") " Jul 24 15:28:20.891827 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.891492 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dk79\" (UniqueName: \"kubernetes.io/projected/c90d8042-e7f0-4dc2-97fb-a634a6740206-kube-api-access-8dk79\") pod \"c90d8042-e7f0-4dc2-97fb-a634a6740206\" (UID: \"c90d8042-e7f0-4dc2-97fb-a634a6740206\") " Jul 24 15:28:20.891946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.891925 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90d8042-e7f0-4dc2-97fb-a634a6740206-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c90d8042-e7f0-4dc2-97fb-a634a6740206" (UID: "c90d8042-e7f0-4dc2-97fb-a634a6740206"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:28:20.893736 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.893717 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c90d8042-e7f0-4dc2-97fb-a634a6740206-kube-api-access-8dk79" (OuterVolumeSpecName: "kube-api-access-8dk79") pod "c90d8042-e7f0-4dc2-97fb-a634a6740206" (UID: "c90d8042-e7f0-4dc2-97fb-a634a6740206"). InnerVolumeSpecName "kube-api-access-8dk79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:28:20.893787 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.893736 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39be795-0da5-4121-9fac-48b3a565fed6-kube-api-access-9dsz7" (OuterVolumeSpecName: "kube-api-access-9dsz7") pod "f39be795-0da5-4121-9fac-48b3a565fed6" (UID: "f39be795-0da5-4121-9fac-48b3a565fed6"). InnerVolumeSpecName "kube-api-access-9dsz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:28:20.950668 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.950612 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:20.957691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.957670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:20.964748 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.964725 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:20.993525 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.993234 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dk79\" (UniqueName: \"kubernetes.io/projected/c90d8042-e7f0-4dc2-97fb-a634a6740206-kube-api-access-8dk79\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:20.993525 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.993270 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c90d8042-e7f0-4dc2-97fb-a634a6740206-extensions-socket-volume\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:20.993525 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:20.993287 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dsz7\" (UniqueName: \"kubernetes.io/projected/f39be795-0da5-4121-9fac-48b3a565fed6-kube-api-access-9dsz7\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:21.115962 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.115911 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x"] Jul 24 15:28:21.121483 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:21.121457 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8f0c29_f4fa_47ce_8d62_ffd4252675c8.slice/crio-aa949d944a4302069826b0993badc92cdc8ce25aa748e4dd3bd4e5c5d431879d WatchSource:0}: Error finding container aa949d944a4302069826b0993badc92cdc8ce25aa748e4dd3bd4e5c5d431879d: Status 404 returned error can't find the container with id aa949d944a4302069826b0993badc92cdc8ce25aa748e4dd3bd4e5c5d431879d Jul 24 15:28:21.348675 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.348566 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh"] Jul 24 15:28:21.350621 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.350582 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx"] Jul 24 15:28:21.352573 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:21.352543 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0654f06f_5299_46a5_838b_ccbc00756c9a.slice/crio-4e8e97c387910c2c6b0808b013f56da40bbdfea4280db5122294e25e4804b397 WatchSource:0}: Error finding container 4e8e97c387910c2c6b0808b013f56da40bbdfea4280db5122294e25e4804b397: Status 404 returned error can't find the container with id 4e8e97c387910c2c6b0808b013f56da40bbdfea4280db5122294e25e4804b397 Jul 24 15:28:21.353182 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:21.353158 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcbc374_1a6e_4bf8_89b9_4191b8d09d2b.slice/crio-4f5faf8c43a1c0a7957e680bf4a683fadaacdafbec13f5cbcf979447c05d2471 WatchSource:0}: Error finding container 4f5faf8c43a1c0a7957e680bf4a683fadaacdafbec13f5cbcf979447c05d2471: Status 404 returned error can't find the container with id 4f5faf8c43a1c0a7957e680bf4a683fadaacdafbec13f5cbcf979447c05d2471 Jul 24 15:28:21.758444 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.758412 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" event={"ID":"0654f06f-5299-46a5-838b-ccbc00756c9a","Type":"ContainerStarted","Data":"b06a7445ba40d78ad62994601a5cde994e7fb86b1d02a791606899907f5d1d03"} Jul 24 15:28:21.758677 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.758459 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" event={"ID":"0654f06f-5299-46a5-838b-ccbc00756c9a","Type":"ContainerStarted","Data":"4e8e97c387910c2c6b0808b013f56da40bbdfea4280db5122294e25e4804b397"} Jul 24 15:28:21.758677 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.758503 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:21.760056 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.760034 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" event={"ID":"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8","Type":"ContainerStarted","Data":"0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210"} Jul 24 15:28:21.760153 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.760078 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:21.760153 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.760103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" event={"ID":"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8","Type":"ContainerStarted","Data":"aa949d944a4302069826b0993badc92cdc8ce25aa748e4dd3bd4e5c5d431879d"} Jul 24 15:28:21.760599 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:21.760568 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:21.761419 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.761401 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:21.761419 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.761410 2564 scope.go:117] "RemoveContainer" containerID="014dda151b02d2cd2d0610e3fc839fe1131bf1ee2cd7aded51e73a4992b1f145" Jul 24 15:28:21.762567 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:21.762544 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:21.763312 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.763288 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" event={"ID":"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b","Type":"ContainerStarted","Data":"f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64"} Jul 24 15:28:21.763447 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.763326 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" event={"ID":"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b","Type":"ContainerStarted","Data":"4f5faf8c43a1c0a7957e680bf4a683fadaacdafbec13f5cbcf979447c05d2471"} Jul 24 15:28:21.763447 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.763372 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:21.764661 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.764633 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:21.766545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.766509 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" containerName="manager" containerID="cri-o://02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580" gracePeriod=2 Jul 24 15:28:21.772491 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.772475 2564 scope.go:117] "RemoveContainer" containerID="4aa013466e8217f4061f89cf13e70c42d572e31925751a7bc4571e9403b78843" Jul 24 15:28:21.795679 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.795573 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" podStartSLOduration=1.7955499499999998 podStartE2EDuration="1.79554995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:21.793868848 +0000 UTC m=+503.717794159" watchObservedRunningTime="2026-07-24 15:28:21.79554995 +0000 UTC m=+503.719475258" Jul 24 15:28:21.815780 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.815728 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" podStartSLOduration=1.8157092700000002 podStartE2EDuration="1.81570927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:21.81531016 +0000 UTC m=+503.739235518" watchObservedRunningTime="2026-07-24 15:28:21.81570927 +0000 UTC m=+503.739634578" Jul 24 15:28:21.817377 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:21.817348 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:21.819272 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:21.819247 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:21.842341 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:21.842297 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" podStartSLOduration=1.842284321 podStartE2EDuration="1.842284321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:21.840987725 +0000 UTC m=+503.764913049" watchObservedRunningTime="2026-07-24 15:28:21.842284321 +0000 UTC m=+503.766209630" Jul 24 15:28:21.843238 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:21.843219 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:22.017301 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.017244 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:22.019411 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:22.019387 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:22.021370 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:22.021347 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-frqbg\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-frqbg" Jul 24 15:28:22.023309 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:22.023288 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cpjkc\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cpjkc" Jul 24 15:28:22.103815 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.103783 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2bd7af7-474a-4b3c-9419-03ecf9691058-extensions-socket-volume\") pod \"a2bd7af7-474a-4b3c-9419-03ecf9691058\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " Jul 24 15:28:22.103815 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.103816 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2744\" (UniqueName: \"kubernetes.io/projected/a2bd7af7-474a-4b3c-9419-03ecf9691058-kube-api-access-w2744\") pod \"a2bd7af7-474a-4b3c-9419-03ecf9691058\" (UID: \"a2bd7af7-474a-4b3c-9419-03ecf9691058\") " Jul 24 15:28:22.104037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.104016 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2bd7af7-474a-4b3c-9419-03ecf9691058-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a2bd7af7-474a-4b3c-9419-03ecf9691058" (UID: "a2bd7af7-474a-4b3c-9419-03ecf9691058"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:28:22.106107 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.106084 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2bd7af7-474a-4b3c-9419-03ecf9691058-kube-api-access-w2744" (OuterVolumeSpecName: "kube-api-access-w2744") pod "a2bd7af7-474a-4b3c-9419-03ecf9691058" (UID: "a2bd7af7-474a-4b3c-9419-03ecf9691058"). InnerVolumeSpecName "kube-api-access-w2744". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:28:22.205435 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.205396 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2bd7af7-474a-4b3c-9419-03ecf9691058-extensions-socket-volume\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:22.205435 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.205428 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2744\" (UniqueName: \"kubernetes.io/projected/a2bd7af7-474a-4b3c-9419-03ecf9691058-kube-api-access-w2744\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:22.533943 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.533908 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" path="/var/lib/kubelet/pods/a2bd7af7-474a-4b3c-9419-03ecf9691058/volumes" Jul 24 15:28:22.534272 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.534259 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c90d8042-e7f0-4dc2-97fb-a634a6740206" path="/var/lib/kubelet/pods/c90d8042-e7f0-4dc2-97fb-a634a6740206/volumes" Jul 24 15:28:22.534535 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.534524 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39be795-0da5-4121-9fac-48b3a565fed6" path="/var/lib/kubelet/pods/f39be795-0da5-4121-9fac-48b3a565fed6/volumes" Jul 24 15:28:22.774796 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.774762 2564 generic.go:356] "Generic (PLEG): container finished" podID="a2bd7af7-474a-4b3c-9419-03ecf9691058" containerID="02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580" exitCode=2 Jul 24 15:28:22.774992 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.774804 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:22.774992 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.774874 2564 scope.go:117] "RemoveContainer" containerID="02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580" Jul 24 15:28:22.779613 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:22.779580 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rzxqn\" is forbidden: User \"system:node:ip-10-0-133-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-102.ec2.internal' and this object" podUID="a2bd7af7-474a-4b3c-9419-03ecf9691058" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rzxqn" Jul 24 15:28:22.783375 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.783355 2564 scope.go:117] "RemoveContainer" containerID="02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580" Jul 24 15:28:22.783675 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:22.783641 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580\": container with ID starting with 02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580 not found: ID does not exist" containerID="02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580" Jul 24 15:28:22.783789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:22.783682 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580"} err="failed to get container status \"02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580\": rpc error: code = NotFound desc = could not find container \"02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580\": container with ID starting with 02ccba418dd6d4dcaecfd9500c96bff65eae3bb34e4e01a1c5a49226f1538580 not found: ID does not exist" Jul 24 15:28:32.777380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:32.777350 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:32.777868 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:32.777430 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:32.777868 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:32.777590 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6nchh" Jul 24 15:28:32.852969 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:32.852936 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx"] Jul 24 15:28:32.853208 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:32.853171 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" podUID="fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" containerName="manager" containerID="cri-o://f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64" gracePeriod=10 Jul 24 15:28:33.085424 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.085398 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz"] Jul 24 15:28:33.088226 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.088207 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.101938 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.101915 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz"] Jul 24 15:28:33.106874 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.106850 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:33.202241 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.202214 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-extensions-socket-volume\") pod \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " Jul 24 15:28:33.202455 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.202264 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p69gg\" (UniqueName: \"kubernetes.io/projected/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-kube-api-access-p69gg\") pod \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\" (UID: \"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b\") " Jul 24 15:28:33.202525 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.202480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c534884d-c415-4c4f-8da3-89b0c43ffae4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xk4nz\" (UID: \"c534884d-c415-4c4f-8da3-89b0c43ffae4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.202597 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.202547 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nftj\" (UniqueName: \"kubernetes.io/projected/c534884d-c415-4c4f-8da3-89b0c43ffae4-kube-api-access-8nftj\") pod \"kuadrant-operator-controller-manager-86878c769f-xk4nz\" (UID: \"c534884d-c415-4c4f-8da3-89b0c43ffae4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.202688 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.202638 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" (UID: "fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:28:33.204727 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.204701 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-kube-api-access-p69gg" (OuterVolumeSpecName: "kube-api-access-p69gg") pod "fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" (UID: "fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b"). InnerVolumeSpecName "kube-api-access-p69gg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:28:33.303331 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.303297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c534884d-c415-4c4f-8da3-89b0c43ffae4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xk4nz\" (UID: \"c534884d-c415-4c4f-8da3-89b0c43ffae4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.303501 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.303373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nftj\" (UniqueName: \"kubernetes.io/projected/c534884d-c415-4c4f-8da3-89b0c43ffae4-kube-api-access-8nftj\") pod \"kuadrant-operator-controller-manager-86878c769f-xk4nz\" (UID: \"c534884d-c415-4c4f-8da3-89b0c43ffae4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.303501 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.303423 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-extensions-socket-volume\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:33.303501 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.303433 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p69gg\" (UniqueName: \"kubernetes.io/projected/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b-kube-api-access-p69gg\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:33.303753 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.303730 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c534884d-c415-4c4f-8da3-89b0c43ffae4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xk4nz\" (UID: \"c534884d-c415-4c4f-8da3-89b0c43ffae4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.321219 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.321186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nftj\" (UniqueName: \"kubernetes.io/projected/c534884d-c415-4c4f-8da3-89b0c43ffae4-kube-api-access-8nftj\") pod \"kuadrant-operator-controller-manager-86878c769f-xk4nz\" (UID: \"c534884d-c415-4c4f-8da3-89b0c43ffae4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.413582 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.413503 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.545062 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.545012 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz"] Jul 24 15:28:33.547410 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:33.547385 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc534884d_c415_4c4f_8da3_89b0c43ffae4.slice/crio-e90afd6d22f2f6add95c0c88f28debd175a0141d00182df4e663c0ce2ae1d4fe WatchSource:0}: Error finding container e90afd6d22f2f6add95c0c88f28debd175a0141d00182df4e663c0ce2ae1d4fe: Status 404 returned error can't find the container with id e90afd6d22f2f6add95c0c88f28debd175a0141d00182df4e663c0ce2ae1d4fe Jul 24 15:28:33.834401 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.834368 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" event={"ID":"c534884d-c415-4c4f-8da3-89b0c43ffae4","Type":"ContainerStarted","Data":"718379575dabd5c2b2f7e1dc25e2d097ebcf87cff3d1938e0b5aadad9eb4b917"} Jul 24 15:28:33.834888 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.834539 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:33.834888 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.834598 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" event={"ID":"c534884d-c415-4c4f-8da3-89b0c43ffae4","Type":"ContainerStarted","Data":"e90afd6d22f2f6add95c0c88f28debd175a0141d00182df4e663c0ce2ae1d4fe"} Jul 24 15:28:33.835778 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.835759 2564 generic.go:356] "Generic (PLEG): container finished" podID="fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" containerID="f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64" exitCode=0 Jul 24 15:28:33.835834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.835793 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" event={"ID":"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b","Type":"ContainerDied","Data":"f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64"} Jul 24 15:28:33.835834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.835798 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" Jul 24 15:28:33.835834 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.835823 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx" event={"ID":"fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b","Type":"ContainerDied","Data":"4f5faf8c43a1c0a7957e680bf4a683fadaacdafbec13f5cbcf979447c05d2471"} Jul 24 15:28:33.835944 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.835844 2564 scope.go:117] "RemoveContainer" containerID="f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64" Jul 24 15:28:33.844978 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.844959 2564 scope.go:117] "RemoveContainer" containerID="f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64" Jul 24 15:28:33.845234 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:33.845217 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64\": container with ID starting with f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64 not found: ID does not exist" containerID="f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64" Jul 24 15:28:33.845299 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.845260 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64"} err="failed to get container status \"f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64\": rpc error: code = NotFound desc = could not find container \"f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64\": container with ID starting with f8b4e599bbb79b420518da82abf6c28bf57cebcfb724cbeb5edb24c5d2104e64 not found: ID does not exist" Jul 24 15:28:33.863421 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.863372 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" podStartSLOduration=0.863357572 podStartE2EDuration="863.357572ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:33.861746984 +0000 UTC m=+515.785672309" watchObservedRunningTime="2026-07-24 15:28:33.863357572 +0000 UTC m=+515.787282881" Jul 24 15:28:33.911721 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.911688 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx"] Jul 24 15:28:33.926996 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:33.926950 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-46jxx"] Jul 24 15:28:34.534217 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:34.534178 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" path="/var/lib/kubelet/pods/fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b/volumes" Jul 24 15:28:44.844316 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:44.844287 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xk4nz" Jul 24 15:28:44.915082 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:44.915051 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x"] Jul 24 15:28:44.915346 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:44.915322 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" podUID="2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" containerName="manager" containerID="cri-o://0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210" gracePeriod=10 Jul 24 15:28:45.161419 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.161393 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:45.308214 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.308165 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn6ld\" (UniqueName: \"kubernetes.io/projected/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-kube-api-access-sn6ld\") pod \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " Jul 24 15:28:45.308406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.308240 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-extensions-socket-volume\") pod \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\" (UID: \"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8\") " Jul 24 15:28:45.308620 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.308590 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" (UID: "2c8f0c29-f4fa-47ce-8d62-ffd4252675c8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:28:45.310433 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.310416 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-kube-api-access-sn6ld" (OuterVolumeSpecName: "kube-api-access-sn6ld") pod "2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" (UID: "2c8f0c29-f4fa-47ce-8d62-ffd4252675c8"). InnerVolumeSpecName "kube-api-access-sn6ld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:28:45.409501 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.409410 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn6ld\" (UniqueName: \"kubernetes.io/projected/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-kube-api-access-sn6ld\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:45.409501 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.409449 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8-extensions-socket-volume\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:28:45.891768 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.891732 2564 generic.go:356] "Generic (PLEG): container finished" podID="2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" containerID="0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210" exitCode=0 Jul 24 15:28:45.892186 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.891775 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" Jul 24 15:28:45.892186 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.891822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" event={"ID":"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8","Type":"ContainerDied","Data":"0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210"} Jul 24 15:28:45.892186 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.891874 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x" event={"ID":"2c8f0c29-f4fa-47ce-8d62-ffd4252675c8","Type":"ContainerDied","Data":"aa949d944a4302069826b0993badc92cdc8ce25aa748e4dd3bd4e5c5d431879d"} Jul 24 15:28:45.892186 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.891897 2564 scope.go:117] "RemoveContainer" containerID="0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210" Jul 24 15:28:45.901338 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.901318 2564 scope.go:117] "RemoveContainer" containerID="0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210" Jul 24 15:28:45.901596 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:28:45.901579 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210\": container with ID starting with 0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210 not found: ID does not exist" containerID="0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210" Jul 24 15:28:45.901688 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.901603 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210"} err="failed to get container status \"0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210\": rpc error: code = NotFound desc = could not find container \"0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210\": container with ID starting with 0eba99f4cc84f5c5bba95b8590e58ac3763d79134f695063baaabdae969ce210 not found: ID does not exist" Jul 24 15:28:45.916888 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.916868 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x"] Jul 24 15:28:45.925143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:45.925123 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rz87x"] Jul 24 15:28:46.534149 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:46.534116 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" path="/var/lib/kubelet/pods/2c8f0c29-f4fa-47ce-8d62-ffd4252675c8/volumes" Jul 24 15:28:49.290437 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290356 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99"] Jul 24 15:28:49.290798 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290774 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" containerName="manager" Jul 24 15:28:49.290798 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290786 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" containerName="manager" Jul 24 15:28:49.290798 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290797 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" containerName="manager" Jul 24 15:28:49.290893 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290802 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" containerName="manager" Jul 24 15:28:49.290893 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290857 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8f0c29-f4fa-47ce-8d62-ffd4252675c8" containerName="manager" Jul 24 15:28:49.290893 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.290865 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdcbc374-1a6e-4bf8-89b9-4191b8d09d2b" containerName="manager" Jul 24 15:28:49.293581 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.293563 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.295980 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.295960 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wz9nv\"" Jul 24 15:28:49.296226 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.296212 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 15:28:49.306914 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.306895 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99"] Jul 24 15:28:49.446708 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446639 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446708 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vg4q\" (UniqueName: \"kubernetes.io/projected/b29b40b4-1250-4427-9c1a-c5cb6551c32d-kube-api-access-7vg4q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446839 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.446952 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.447240 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.446975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547390 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547316 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547390 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547390 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547693 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547568 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547692 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547717 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vg4q\" (UniqueName: \"kubernetes.io/projected/b29b40b4-1250-4427-9c1a-c5cb6551c32d-kube-api-access-7vg4q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547754 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547803 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547829 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.547957 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.547801 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.548148 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.548079 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.548148 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.548087 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.548529 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.548510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.549857 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.549838 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.550374 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.550357 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.560278 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.560250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b29b40b4-1250-4427-9c1a-c5cb6551c32d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.560278 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.560251 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vg4q\" (UniqueName: \"kubernetes.io/projected/b29b40b4-1250-4427-9c1a-c5cb6551c32d-kube-api-access-7vg4q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fcj99\" (UID: \"b29b40b4-1250-4427-9c1a-c5cb6551c32d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.604011 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.603980 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:49.747515 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.747490 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99"] Jul 24 15:28:49.749412 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:28:49.749386 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29b40b4_1250_4427_9c1a_c5cb6551c32d.slice/crio-934456e4e15f247c95f83f45434948f6f63e2e8f9f5f5ea3cf8460bb0c4db75a WatchSource:0}: Error finding container 934456e4e15f247c95f83f45434948f6f63e2e8f9f5f5ea3cf8460bb0c4db75a: Status 404 returned error can't find the container with id 934456e4e15f247c95f83f45434948f6f63e2e8f9f5f5ea3cf8460bb0c4db75a Jul 24 15:28:49.914629 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:49.914545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" event={"ID":"b29b40b4-1250-4427-9c1a-c5cb6551c32d","Type":"ContainerStarted","Data":"934456e4e15f247c95f83f45434948f6f63e2e8f9f5f5ea3cf8460bb0c4db75a"} Jul 24 15:28:52.639905 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:52.639873 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:28:52.640195 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:52.639939 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:28:52.640195 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:52.639971 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:28:52.931417 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:52.931329 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" event={"ID":"b29b40b4-1250-4427-9c1a-c5cb6551c32d","Type":"ContainerStarted","Data":"900eb0b3516d93f5b69b12d3243b408ddebfdedf5baee075ce601088c2b2cc20"} Jul 24 15:28:52.958935 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:52.958839 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" podStartSLOduration=1.070468779 podStartE2EDuration="3.958824751s" totalImagesPullingTime="2.888355972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:28:52.957613744 +0000 UTC m=+534.881539092" watchObservedRunningTime="2026-07-24 15:28:52.958824751 +0000 UTC m=+534.882750058" Jul 24 15:28:53.604159 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:53.604125 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:53.608807 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:53.608769 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:53.936715 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:53.936626 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:28:53.937554 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:28:53.937536 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fcj99" Jul 24 15:29:05.202122 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.202089 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:05.207062 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.207044 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.209932 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.209908 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-b99vp\"" Jul 24 15:29:05.210045 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.209952 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 24 15:29:05.216954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.216928 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:05.284591 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.284543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj6pl\" (UniqueName: \"kubernetes.io/projected/3f00cc1e-b867-4fb3-bcb2-c7e598517563-kube-api-access-sj6pl\") pod \"limitador-limitador-64c94fd64f-4v5cj\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.285010 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.284989 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f00cc1e-b867-4fb3-bcb2-c7e598517563-config-file\") pod \"limitador-limitador-64c94fd64f-4v5cj\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.322680 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.322574 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:05.386297 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.386257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f00cc1e-b867-4fb3-bcb2-c7e598517563-config-file\") pod \"limitador-limitador-64c94fd64f-4v5cj\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.386515 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.386347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj6pl\" (UniqueName: \"kubernetes.io/projected/3f00cc1e-b867-4fb3-bcb2-c7e598517563-kube-api-access-sj6pl\") pod \"limitador-limitador-64c94fd64f-4v5cj\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.387137 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.387116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f00cc1e-b867-4fb3-bcb2-c7e598517563-config-file\") pod \"limitador-limitador-64c94fd64f-4v5cj\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.396158 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.396130 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj6pl\" (UniqueName: \"kubernetes.io/projected/3f00cc1e-b867-4fb3-bcb2-c7e598517563-kube-api-access-sj6pl\") pod \"limitador-limitador-64c94fd64f-4v5cj\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.518032 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.517949 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:05.652904 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.652865 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:05.655613 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:29:05.655589 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f00cc1e_b867_4fb3_bcb2_c7e598517563.slice/crio-8440d46f9fae46931a5a489eb7a8d936f14f92721205accf2e4e205121d505ba WatchSource:0}: Error finding container 8440d46f9fae46931a5a489eb7a8d936f14f92721205accf2e4e205121d505ba: Status 404 returned error can't find the container with id 8440d46f9fae46931a5a489eb7a8d936f14f92721205accf2e4e205121d505ba Jul 24 15:29:05.995105 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:05.995072 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" event={"ID":"3f00cc1e-b867-4fb3-bcb2-c7e598517563","Type":"ContainerStarted","Data":"8440d46f9fae46931a5a489eb7a8d936f14f92721205accf2e4e205121d505ba"} Jul 24 15:29:06.230246 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.230209 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbfnd"] Jul 24 15:29:06.235163 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.235139 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:06.237741 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.237722 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sr8ns\"" Jul 24 15:29:06.243781 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.243761 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbfnd"] Jul 24 15:29:06.293439 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.293370 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjg9c\" (UniqueName: \"kubernetes.io/projected/fc59f63e-e46f-43ca-8313-c2a1600cfc17-kube-api-access-gjg9c\") pod \"authorino-75d556bd77-qbfnd\" (UID: \"fc59f63e-e46f-43ca-8313-c2a1600cfc17\") " pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:06.394371 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.394326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjg9c\" (UniqueName: \"kubernetes.io/projected/fc59f63e-e46f-43ca-8313-c2a1600cfc17-kube-api-access-gjg9c\") pod \"authorino-75d556bd77-qbfnd\" (UID: \"fc59f63e-e46f-43ca-8313-c2a1600cfc17\") " pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:06.404130 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.404060 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjg9c\" (UniqueName: \"kubernetes.io/projected/fc59f63e-e46f-43ca-8313-c2a1600cfc17-kube-api-access-gjg9c\") pod \"authorino-75d556bd77-qbfnd\" (UID: \"fc59f63e-e46f-43ca-8313-c2a1600cfc17\") " pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:06.545631 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.545543 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:06.728727 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:06.728682 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbfnd"] Jul 24 15:29:06.732168 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:29:06.732093 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc59f63e_e46f_43ca_8313_c2a1600cfc17.slice/crio-96e55f06c9afdefd2be1fc7f82c6ed937af2d885da273885066389211efd7a9c WatchSource:0}: Error finding container 96e55f06c9afdefd2be1fc7f82c6ed937af2d885da273885066389211efd7a9c: Status 404 returned error can't find the container with id 96e55f06c9afdefd2be1fc7f82c6ed937af2d885da273885066389211efd7a9c Jul 24 15:29:07.001177 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:07.001147 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbfnd" event={"ID":"fc59f63e-e46f-43ca-8313-c2a1600cfc17","Type":"ContainerStarted","Data":"96e55f06c9afdefd2be1fc7f82c6ed937af2d885da273885066389211efd7a9c"} Jul 24 15:29:09.016189 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:09.016148 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" event={"ID":"3f00cc1e-b867-4fb3-bcb2-c7e598517563","Type":"ContainerStarted","Data":"e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060"} Jul 24 15:29:09.016691 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:09.016362 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:09.035412 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:09.035357 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" podStartSLOduration=1.205865967 podStartE2EDuration="4.035336017s" totalImagesPullingTime="2.82947005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:29:09.033396685 +0000 UTC m=+550.957322019" watchObservedRunningTime="2026-07-24 15:29:09.035336017 +0000 UTC m=+550.959261326" Jul 24 15:29:11.028121 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:11.028085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbfnd" event={"ID":"fc59f63e-e46f-43ca-8313-c2a1600cfc17","Type":"ContainerStarted","Data":"4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9"} Jul 24 15:29:11.045452 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:11.045396 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qbfnd" podStartSLOduration=1.214553007 podStartE2EDuration="5.045381774s" totalImagesPullingTime="3.830828767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:29:11.043855282 +0000 UTC m=+552.967780589" watchObservedRunningTime="2026-07-24 15:29:11.045381774 +0000 UTC m=+552.969307083" Jul 24 15:29:20.022931 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:20.022899 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:22.721362 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:22.721330 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:22.721789 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:22.721511 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" podUID="3f00cc1e-b867-4fb3-bcb2-c7e598517563" containerName="limitador" containerID="cri-o://e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060" gracePeriod=30 Jul 24 15:29:23.273318 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.273293 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:23.363506 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.363405 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj6pl\" (UniqueName: \"kubernetes.io/projected/3f00cc1e-b867-4fb3-bcb2-c7e598517563-kube-api-access-sj6pl\") pod \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " Jul 24 15:29:23.363506 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.363467 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f00cc1e-b867-4fb3-bcb2-c7e598517563-config-file\") pod \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\" (UID: \"3f00cc1e-b867-4fb3-bcb2-c7e598517563\") " Jul 24 15:29:23.363890 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.363868 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f00cc1e-b867-4fb3-bcb2-c7e598517563-config-file" (OuterVolumeSpecName: "config-file") pod "3f00cc1e-b867-4fb3-bcb2-c7e598517563" (UID: "3f00cc1e-b867-4fb3-bcb2-c7e598517563"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:29:23.365680 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.365658 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f00cc1e-b867-4fb3-bcb2-c7e598517563-kube-api-access-sj6pl" (OuterVolumeSpecName: "kube-api-access-sj6pl") pod "3f00cc1e-b867-4fb3-bcb2-c7e598517563" (UID: "3f00cc1e-b867-4fb3-bcb2-c7e598517563"). InnerVolumeSpecName "kube-api-access-sj6pl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:29:23.465024 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.464991 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj6pl\" (UniqueName: \"kubernetes.io/projected/3f00cc1e-b867-4fb3-bcb2-c7e598517563-kube-api-access-sj6pl\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:29:23.465024 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:23.465017 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f00cc1e-b867-4fb3-bcb2-c7e598517563-config-file\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:29:24.088374 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.088342 2564 generic.go:356] "Generic (PLEG): container finished" podID="3f00cc1e-b867-4fb3-bcb2-c7e598517563" containerID="e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060" exitCode=0 Jul 24 15:29:24.088861 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.088390 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" Jul 24 15:29:24.088861 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.088419 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" event={"ID":"3f00cc1e-b867-4fb3-bcb2-c7e598517563","Type":"ContainerDied","Data":"e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060"} Jul 24 15:29:24.088861 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.088474 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4v5cj" event={"ID":"3f00cc1e-b867-4fb3-bcb2-c7e598517563","Type":"ContainerDied","Data":"8440d46f9fae46931a5a489eb7a8d936f14f92721205accf2e4e205121d505ba"} Jul 24 15:29:24.088861 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.088498 2564 scope.go:117] "RemoveContainer" containerID="e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060" Jul 24 15:29:24.097382 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.097361 2564 scope.go:117] "RemoveContainer" containerID="e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060" Jul 24 15:29:24.097634 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:29:24.097610 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060\": container with ID starting with e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060 not found: ID does not exist" containerID="e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060" Jul 24 15:29:24.097733 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.097661 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060"} err="failed to get container status \"e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060\": rpc error: code = NotFound desc = could not find container \"e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060\": container with ID starting with e64200b0956e8e0cff8eda9f3727af6b88a4f62f2878f16dcea6aee054622060 not found: ID does not exist" Jul 24 15:29:24.110116 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.110090 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:24.114603 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.114579 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4v5cj"] Jul 24 15:29:24.534018 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:24.533984 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f00cc1e-b867-4fb3-bcb2-c7e598517563" path="/var/lib/kubelet/pods/3f00cc1e-b867-4fb3-bcb2-c7e598517563/volumes" Jul 24 15:29:26.051922 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.051887 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-gldg6"] Jul 24 15:29:26.052276 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.052260 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f00cc1e-b867-4fb3-bcb2-c7e598517563" containerName="limitador" Jul 24 15:29:26.052276 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.052275 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f00cc1e-b867-4fb3-bcb2-c7e598517563" containerName="limitador" Jul 24 15:29:26.052349 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.052341 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f00cc1e-b867-4fb3-bcb2-c7e598517563" containerName="limitador" Jul 24 15:29:26.057304 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.057282 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.059971 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.059949 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cgsw6\"" Jul 24 15:29:26.060097 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.060021 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 24 15:29:26.060345 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.060327 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 15:29:26.061064 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.061049 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 24 15:29:26.066038 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.066017 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-gldg6"] Jul 24 15:29:26.189670 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.189621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b0f5e6bc-d559-43ec-ad2a-790d55959d9b-data\") pod \"postgres-69fb4594cf-gldg6\" (UID: \"b0f5e6bc-d559-43ec-ad2a-790d55959d9b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.189839 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.189682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2cpj\" (UniqueName: \"kubernetes.io/projected/b0f5e6bc-d559-43ec-ad2a-790d55959d9b-kube-api-access-t2cpj\") pod \"postgres-69fb4594cf-gldg6\" (UID: \"b0f5e6bc-d559-43ec-ad2a-790d55959d9b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.290961 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.290931 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b0f5e6bc-d559-43ec-ad2a-790d55959d9b-data\") pod \"postgres-69fb4594cf-gldg6\" (UID: \"b0f5e6bc-d559-43ec-ad2a-790d55959d9b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.291152 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.290971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2cpj\" (UniqueName: \"kubernetes.io/projected/b0f5e6bc-d559-43ec-ad2a-790d55959d9b-kube-api-access-t2cpj\") pod \"postgres-69fb4594cf-gldg6\" (UID: \"b0f5e6bc-d559-43ec-ad2a-790d55959d9b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.291407 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.291385 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b0f5e6bc-d559-43ec-ad2a-790d55959d9b-data\") pod \"postgres-69fb4594cf-gldg6\" (UID: \"b0f5e6bc-d559-43ec-ad2a-790d55959d9b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.300377 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.300353 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2cpj\" (UniqueName: \"kubernetes.io/projected/b0f5e6bc-d559-43ec-ad2a-790d55959d9b-kube-api-access-t2cpj\") pod \"postgres-69fb4594cf-gldg6\" (UID: \"b0f5e6bc-d559-43ec-ad2a-790d55959d9b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.369241 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.369162 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:26.506722 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:26.506698 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-gldg6"] Jul 24 15:29:26.509735 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:29:26.509707 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f5e6bc_d559_43ec_ad2a_790d55959d9b.slice/crio-7c11e7e5da4117628018a6fd710bbd30ab4f4b1606c04f57a719b4953df773c4 WatchSource:0}: Error finding container 7c11e7e5da4117628018a6fd710bbd30ab4f4b1606c04f57a719b4953df773c4: Status 404 returned error can't find the container with id 7c11e7e5da4117628018a6fd710bbd30ab4f4b1606c04f57a719b4953df773c4 Jul 24 15:29:27.104135 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:27.104098 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" event={"ID":"b0f5e6bc-d559-43ec-ad2a-790d55959d9b","Type":"ContainerStarted","Data":"7c11e7e5da4117628018a6fd710bbd30ab4f4b1606c04f57a719b4953df773c4"} Jul 24 15:29:32.135030 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:32.134999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" event={"ID":"b0f5e6bc-d559-43ec-ad2a-790d55959d9b","Type":"ContainerStarted","Data":"2a9c811dfa14ca63a17f88e16ac4729c8f3e06bcdf1eed6f7de0cd8d6d8462d9"} Jul 24 15:29:32.135497 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:32.135123 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:32.151790 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:32.151720 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" podStartSLOduration=0.720986427 podStartE2EDuration="6.151706111s" totalImagesPullingTime="5.430719684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:29:32.151076189 +0000 UTC m=+574.075001497" watchObservedRunningTime="2026-07-24 15:29:32.151706111 +0000 UTC m=+574.075631419" Jul 24 15:29:38.169586 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:38.169553 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gldg6" Jul 24 15:29:40.865395 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:40.865358 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbfnd"] Jul 24 15:29:40.865808 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:40.865555 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qbfnd" podUID="fc59f63e-e46f-43ca-8313-c2a1600cfc17" containerName="authorino" containerID="cri-o://4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9" gracePeriod=30 Jul 24 15:29:41.127282 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.127224 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:41.184858 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.184819 2564 generic.go:356] "Generic (PLEG): container finished" podID="fc59f63e-e46f-43ca-8313-c2a1600cfc17" containerID="4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9" exitCode=0 Jul 24 15:29:41.184858 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.184867 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qbfnd" Jul 24 15:29:41.185074 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.184900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbfnd" event={"ID":"fc59f63e-e46f-43ca-8313-c2a1600cfc17","Type":"ContainerDied","Data":"4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9"} Jul 24 15:29:41.185074 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.184968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qbfnd" event={"ID":"fc59f63e-e46f-43ca-8313-c2a1600cfc17","Type":"ContainerDied","Data":"96e55f06c9afdefd2be1fc7f82c6ed937af2d885da273885066389211efd7a9c"} Jul 24 15:29:41.185074 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.184991 2564 scope.go:117] "RemoveContainer" containerID="4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9" Jul 24 15:29:41.194146 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.194126 2564 scope.go:117] "RemoveContainer" containerID="4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9" Jul 24 15:29:41.194436 ip-10-0-133-102 kubenswrapper[2564]: E0724 15:29:41.194417 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9\": container with ID starting with 4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9 not found: ID does not exist" containerID="4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9" Jul 24 15:29:41.194509 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.194446 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9"} err="failed to get container status \"4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9\": rpc error: code = NotFound desc = could not find container \"4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9\": container with ID starting with 4dc21cefd36771679db18ef1ed17cadbba75e7de9e39ae18d4cb0ae639eadca9 not found: ID does not exist" Jul 24 15:29:41.216817 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.216793 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjg9c\" (UniqueName: \"kubernetes.io/projected/fc59f63e-e46f-43ca-8313-c2a1600cfc17-kube-api-access-gjg9c\") pod \"fc59f63e-e46f-43ca-8313-c2a1600cfc17\" (UID: \"fc59f63e-e46f-43ca-8313-c2a1600cfc17\") " Jul 24 15:29:41.218996 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.218964 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc59f63e-e46f-43ca-8313-c2a1600cfc17-kube-api-access-gjg9c" (OuterVolumeSpecName: "kube-api-access-gjg9c") pod "fc59f63e-e46f-43ca-8313-c2a1600cfc17" (UID: "fc59f63e-e46f-43ca-8313-c2a1600cfc17"). InnerVolumeSpecName "kube-api-access-gjg9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:29:41.318406 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.318371 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjg9c\" (UniqueName: \"kubernetes.io/projected/fc59f63e-e46f-43ca-8313-c2a1600cfc17-kube-api-access-gjg9c\") on node \"ip-10-0-133-102.ec2.internal\" DevicePath \"\"" Jul 24 15:29:41.506965 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.506927 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbfnd"] Jul 24 15:29:41.510983 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:41.510960 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qbfnd"] Jul 24 15:29:42.534813 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:29:42.534784 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc59f63e-e46f-43ca-8313-c2a1600cfc17" path="/var/lib/kubelet/pods/fc59f63e-e46f-43ca-8313-c2a1600cfc17/volumes" Jul 24 15:40:15.306836 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:15.306807 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c97lt_c2b03f75-deaf-4b23-9fa3-1bc5a362a003/manager/0.log" Jul 24 15:40:15.430333 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:15.430303 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zmkdg_caacabef-2f1f-4aaa-91da-b0f8cdf1b322/manager/0.log" Jul 24 15:40:15.690380 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:15.690344 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m5rpq_43851c5b-baa5-4c82-95cb-43c7fcb89dea/server/0.log" Jul 24 15:40:15.817604 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:15.817574 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zpwsm_101c1923-d051-4a02-bf28-99b66bc1029b/manager/0.log" Jul 24 15:40:16.212343 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:16.212318 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f445d9dcf-stggk_8571bdb3-1773-450e-bd43-7f32bd7e52e0/copy-manifests/0.log" Jul 24 15:40:16.226319 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:16.226292 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f445d9dcf-stggk_8571bdb3-1773-450e-bd43-7f32bd7e52e0/manager/0.log" Jul 24 15:40:16.603400 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:16.603324 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-gldg6_b0f5e6bc-d559-43ec-ad2a-790d55959d9b/postgres/0.log" Jul 24 15:40:18.119845 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:18.119819 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-742nh_f38d2018-77f6-47e4-a25c-dd14651f00ed/manager/0.log" Jul 24 15:40:18.238800 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:18.238771 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-g2dnh_5a205938-52a1-47ba-bfca-0703c77fb2fa/manager/0.log" Jul 24 15:40:18.473464 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:18.473420 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-sjnkn_37c57504-b8be-4d0b-8f79-c35c358c284a/registry-server/0.log" Jul 24 15:40:18.599585 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:18.599545 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xk4nz_c534884d-c415-4c4f-8da3-89b0c43ffae4/manager/0.log" Jul 24 15:40:18.832240 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:18.832161 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6nchh_0654f06f-5299-46a5-838b-ccbc00756c9a/manager/0.log" Jul 24 15:40:19.602790 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:19.602750 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57b97974c4-lwhc9_f69159b4-7150-4e9e-91d4-71ac49e5be4a/kube-auth-proxy/0.log" Jul 24 15:40:19.718107 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:19.718075 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fcj99_b29b40b4-1250-4427-9c1a-c5cb6551c32d/istio-proxy/0.log" Jul 24 15:40:25.955325 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.955292 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4vxxd/must-gather-md8sk"] Jul 24 15:40:25.955988 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.955722 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc59f63e-e46f-43ca-8313-c2a1600cfc17" containerName="authorino" Jul 24 15:40:25.955988 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.955740 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc59f63e-e46f-43ca-8313-c2a1600cfc17" containerName="authorino" Jul 24 15:40:25.955988 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.955855 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc59f63e-e46f-43ca-8313-c2a1600cfc17" containerName="authorino" Jul 24 15:40:25.958881 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.958862 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:25.961556 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.961535 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4vxxd\"/\"default-dockercfg-dl2lv\"" Jul 24 15:40:25.961630 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.961558 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4vxxd\"/\"kube-root-ca.crt\"" Jul 24 15:40:25.962590 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.962569 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4vxxd\"/\"openshift-service-ca.crt\"" Jul 24 15:40:25.976275 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:25.974195 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4vxxd/must-gather-md8sk"] Jul 24 15:40:26.056914 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.056883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssctr\" (UniqueName: \"kubernetes.io/projected/3f5b3887-3112-4c87-8777-5125a173395a-kube-api-access-ssctr\") pod \"must-gather-md8sk\" (UID: \"3f5b3887-3112-4c87-8777-5125a173395a\") " pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.057073 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.056959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3f5b3887-3112-4c87-8777-5125a173395a-must-gather-output\") pod \"must-gather-md8sk\" (UID: \"3f5b3887-3112-4c87-8777-5125a173395a\") " pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.158028 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.157984 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3f5b3887-3112-4c87-8777-5125a173395a-must-gather-output\") pod \"must-gather-md8sk\" (UID: \"3f5b3887-3112-4c87-8777-5125a173395a\") " pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.158191 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.158044 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssctr\" (UniqueName: \"kubernetes.io/projected/3f5b3887-3112-4c87-8777-5125a173395a-kube-api-access-ssctr\") pod \"must-gather-md8sk\" (UID: \"3f5b3887-3112-4c87-8777-5125a173395a\") " pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.158346 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.158326 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3f5b3887-3112-4c87-8777-5125a173395a-must-gather-output\") pod \"must-gather-md8sk\" (UID: \"3f5b3887-3112-4c87-8777-5125a173395a\") " pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.169024 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.168998 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssctr\" (UniqueName: \"kubernetes.io/projected/3f5b3887-3112-4c87-8777-5125a173395a-kube-api-access-ssctr\") pod \"must-gather-md8sk\" (UID: \"3f5b3887-3112-4c87-8777-5125a173395a\") " pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.268323 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.268233 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4vxxd/must-gather-md8sk" Jul 24 15:40:26.404456 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.404430 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4vxxd/must-gather-md8sk"] Jul 24 15:40:26.406743 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:40:26.406707 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f5b3887_3112_4c87_8777_5125a173395a.slice/crio-05e9a668e4d5eff51cc5ccb15e4a352bbecd3f956c4cc4d8e5c050231580bd81 WatchSource:0}: Error finding container 05e9a668e4d5eff51cc5ccb15e4a352bbecd3f956c4cc4d8e5c050231580bd81: Status 404 returned error can't find the container with id 05e9a668e4d5eff51cc5ccb15e4a352bbecd3f956c4cc4d8e5c050231580bd81 Jul 24 15:40:26.408571 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:26.408554 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:40:27.141447 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:27.141422 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vxxd/must-gather-md8sk" event={"ID":"3f5b3887-3112-4c87-8777-5125a173395a","Type":"ContainerStarted","Data":"05e9a668e4d5eff51cc5ccb15e4a352bbecd3f956c4cc4d8e5c050231580bd81"} Jul 24 15:40:28.153193 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:28.153146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vxxd/must-gather-md8sk" event={"ID":"3f5b3887-3112-4c87-8777-5125a173395a","Type":"ContainerStarted","Data":"065d39acc13d2f8af33fa1ac1a37d3319bc4c8af3f2326e103805aa572487146"} Jul 24 15:40:28.153704 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:28.153207 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vxxd/must-gather-md8sk" event={"ID":"3f5b3887-3112-4c87-8777-5125a173395a","Type":"ContainerStarted","Data":"79b86eb90f6884b15f1071c235d75f74b09dacaadd62abca19e0ad22203369fd"} Jul 24 15:40:28.171636 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:28.171558 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4vxxd/must-gather-md8sk" podStartSLOduration=2.45072992 podStartE2EDuration="3.171535495s" totalImagesPullingTime="720.805575ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:40:28.169389969 +0000 UTC m=+1230.093315280" watchObservedRunningTime="2026-07-24 15:40:28.171535495 +0000 UTC m=+1230.095460805" Jul 24 15:40:28.710027 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:28.709997 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-24625_4d8711f9-2375-40fb-8a92-ba56c25afc17/global-pull-secret-syncer/0.log" Jul 24 15:40:28.925600 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:28.925565 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cwk2v_3313863b-64c5-4ddd-b6d4-bd1401402c52/konnectivity-agent/0.log" Jul 24 15:40:28.973959 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:28.973876 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-102.ec2.internal_414259330d98a4b19eb5aa9f7c16c705/haproxy/0.log" Jul 24 15:40:33.927128 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:33.927084 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-742nh_f38d2018-77f6-47e4-a25c-dd14651f00ed/manager/0.log" Jul 24 15:40:33.981675 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:33.981057 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-g2dnh_5a205938-52a1-47ba-bfca-0703c77fb2fa/manager/0.log" Jul 24 15:40:34.050801 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:34.050767 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-sjnkn_37c57504-b8be-4d0b-8f79-c35c358c284a/registry-server/0.log" Jul 24 15:40:34.084197 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:34.084159 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xk4nz_c534884d-c415-4c4f-8da3-89b0c43ffae4/manager/0.log" Jul 24 15:40:34.141399 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:34.141365 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6nchh_0654f06f-5299-46a5-838b-ccbc00756c9a/manager/0.log" Jul 24 15:40:35.684496 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.684455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/alertmanager/0.log" Jul 24 15:40:35.712635 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.712607 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/config-reloader/0.log" Jul 24 15:40:35.741914 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.741841 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/kube-rbac-proxy-web/0.log" Jul 24 15:40:35.767087 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.767033 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/kube-rbac-proxy/0.log" Jul 24 15:40:35.791722 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.791674 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/kube-rbac-proxy-metric/0.log" Jul 24 15:40:35.815488 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.815452 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/prom-label-proxy/0.log" Jul 24 15:40:35.840420 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:35.840381 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4daa1da-d627-412c-9f76-08439cacd9c6/init-config-reloader/0.log" Jul 24 15:40:36.191537 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.191427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d69kr_9fb4cadc-d7a3-42a0-a220-9cd17b3088f5/node-exporter/0.log" Jul 24 15:40:36.231741 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.231621 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d69kr_9fb4cadc-d7a3-42a0-a220-9cd17b3088f5/kube-rbac-proxy/0.log" Jul 24 15:40:36.265398 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.265363 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d69kr_9fb4cadc-d7a3-42a0-a220-9cd17b3088f5/init-textfile/0.log" Jul 24 15:40:36.609849 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.609730 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/prometheus/0.log" Jul 24 15:40:36.645770 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.645737 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/config-reloader/0.log" Jul 24 15:40:36.677692 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.677641 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/thanos-sidecar/0.log" Jul 24 15:40:36.705125 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.705104 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/kube-rbac-proxy-web/0.log" Jul 24 15:40:36.736195 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.736170 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/kube-rbac-proxy/0.log" Jul 24 15:40:36.773111 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.773070 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/kube-rbac-proxy-thanos/0.log" Jul 24 15:40:36.802694 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:36.802641 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cbbbdfb3-5822-4fe9-9507-d3cdc04d1325/init-config-reloader/0.log" Jul 24 15:40:38.345761 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.345718 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-b87wr_07aeeb86-6f77-4012-9f0f-4c6c16c170a2/networking-console-plugin/0.log" Jul 24 15:40:38.815844 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.815811 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9"] Jul 24 15:40:38.823618 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.823594 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.829309 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.829279 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9"] Jul 24 15:40:38.887469 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.887438 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-lib-modules\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.887685 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.887497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-sys\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.887685 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.887569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-proc\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.887833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.887731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfkg5\" (UniqueName: \"kubernetes.io/projected/df24f929-815a-4e92-a136-f8cc0350edf9-kube-api-access-wfkg5\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.887833 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.887797 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-podres\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.989304 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.989248 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-sys\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.989545 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.989521 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-sys\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.989677 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.989538 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-proc\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.989867 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.989852 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfkg5\" (UniqueName: \"kubernetes.io/projected/df24f929-815a-4e92-a136-f8cc0350edf9-kube-api-access-wfkg5\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.990321 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.990304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-podres\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.990606 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.990540 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-podres\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.990857 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.989693 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-proc\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.990954 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.990723 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-lib-modules\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.991074 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.990807 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df24f929-815a-4e92-a136-f8cc0350edf9-lib-modules\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:38.998666 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:38.998620 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfkg5\" (UniqueName: \"kubernetes.io/projected/df24f929-815a-4e92-a136-f8cc0350edf9-kube-api-access-wfkg5\") pod \"perf-node-gather-daemonset-s65f9\" (UID: \"df24f929-815a-4e92-a136-f8cc0350edf9\") " pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:39.140426 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:39.140338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:39.317211 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:39.317174 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9"] Jul 24 15:40:39.321039 ip-10-0-133-102 kubenswrapper[2564]: W0724 15:40:39.321007 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddf24f929_815a_4e92_a136_f8cc0350edf9.slice/crio-7fd122260e16421b4e584c96cf3d10a5c5fcda8143f86da263ab6c58074763e4 WatchSource:0}: Error finding container 7fd122260e16421b4e584c96cf3d10a5c5fcda8143f86da263ab6c58074763e4: Status 404 returned error can't find the container with id 7fd122260e16421b4e584c96cf3d10a5c5fcda8143f86da263ab6c58074763e4 Jul 24 15:40:40.241283 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:40.241235 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" event={"ID":"df24f929-815a-4e92-a136-f8cc0350edf9","Type":"ContainerStarted","Data":"c0c68a28075b4c41602076ce5540761de55a2213931a35b2476ce1a6eb5fe410"} Jul 24 15:40:40.241784 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:40.241311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" event={"ID":"df24f929-815a-4e92-a136-f8cc0350edf9","Type":"ContainerStarted","Data":"7fd122260e16421b4e584c96cf3d10a5c5fcda8143f86da263ab6c58074763e4"} Jul 24 15:40:40.242060 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:40.242034 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:40.261004 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:40.260932 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" podStartSLOduration=2.260910999 podStartE2EDuration="2.260910999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:40:40.259110339 +0000 UTC m=+1242.183035648" watchObservedRunningTime="2026-07-24 15:40:40.260910999 +0000 UTC m=+1242.184836309" Jul 24 15:40:41.274975 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:41.274949 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7s5hw_b075ac19-067d-4c2c-bce6-7189606d8fdf/dns/0.log" Jul 24 15:40:41.300174 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:41.300147 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7s5hw_b075ac19-067d-4c2c-bce6-7189606d8fdf/kube-rbac-proxy/0.log" Jul 24 15:40:41.489940 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:41.489914 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tn7vd_9d9172f6-f70c-46db-8607-b3413f6afb8d/dns-node-resolver/0.log" Jul 24 15:40:42.064820 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:42.064777 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dhhwh_d2699c8d-2cd9-44e3-9ada-1885e5bc0a47/node-ca/0.log" Jul 24 15:40:43.154355 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:43.154325 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57b97974c4-lwhc9_f69159b4-7150-4e9e-91d4-71ac49e5be4a/kube-auth-proxy/0.log" Jul 24 15:40:43.186235 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:43.186203 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fcj99_b29b40b4-1250-4427-9c1a-c5cb6551c32d/istio-proxy/0.log" Jul 24 15:40:43.849387 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:43.849358 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfq8b_86cae9d7-98c7-468d-9d14-7d368de5e8a8/serve-healthcheck-canary/0.log" Jul 24 15:40:44.466048 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:44.466015 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftppt_092e3a8a-1471-4f54-ab2d-9653c2638d65/kube-rbac-proxy/0.log" Jul 24 15:40:44.494118 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:44.494086 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftppt_092e3a8a-1471-4f54-ab2d-9653c2638d65/exporter/0.log" Jul 24 15:40:44.550620 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:44.550538 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftppt_092e3a8a-1471-4f54-ab2d-9653c2638d65/extractor/0.log" Jul 24 15:40:46.711980 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:46.711951 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c97lt_c2b03f75-deaf-4b23-9fa3-1bc5a362a003/manager/0.log" Jul 24 15:40:46.738926 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:46.738892 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zmkdg_caacabef-2f1f-4aaa-91da-b0f8cdf1b322/manager/0.log" Jul 24 15:40:46.804069 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:46.804043 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m5rpq_43851c5b-baa5-4c82-95cb-43c7fcb89dea/server/0.log" Jul 24 15:40:46.857358 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:46.857320 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zpwsm_101c1923-d051-4a02-bf28-99b66bc1029b/manager/0.log" Jul 24 15:40:47.043392 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:47.043315 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f445d9dcf-stggk_8571bdb3-1773-450e-bd43-7f32bd7e52e0/manager/0.log" Jul 24 15:40:47.068456 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:47.068430 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f445d9dcf-stggk_8571bdb3-1773-450e-bd43-7f32bd7e52e0/copy-manifests/0.log" Jul 24 15:40:47.263143 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:47.263119 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4vxxd/perf-node-gather-daemonset-s65f9" Jul 24 15:40:49.076456 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:49.076425 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57b97974c4-lwhc9_f69159b4-7150-4e9e-91d4-71ac49e5be4a/kube-auth-proxy/0.log" Jul 24 15:40:49.105696 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:49.105668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fcj99_b29b40b4-1250-4427-9c1a-c5cb6551c32d/istio-proxy/0.log" Jul 24 15:40:50.409942 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:50.409910 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57b97974c4-lwhc9_f69159b4-7150-4e9e-91d4-71ac49e5be4a/kube-auth-proxy/0.log" Jul 24 15:40:50.429037 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:50.429008 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fcj99_b29b40b4-1250-4427-9c1a-c5cb6551c32d/istio-proxy/0.log" Jul 24 15:40:53.838874 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:53.838845 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-xvsq6_a4ccc847-ece6-440c-bc4b-329f789260c9/migrator/0.log" Jul 24 15:40:53.862261 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:53.862234 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-xvsq6_a4ccc847-ece6-440c-bc4b-329f789260c9/graceful-termination/0.log" Jul 24 15:40:55.357188 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.357159 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/kube-multus-additional-cni-plugins/0.log" Jul 24 15:40:55.383041 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.383016 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/egress-router-binary-copy/0.log" Jul 24 15:40:55.408946 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.408917 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/cni-plugins/0.log" Jul 24 15:40:55.431826 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.431803 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/bond-cni-plugin/0.log" Jul 24 15:40:55.457565 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.457538 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/routeoverride-cni/0.log" Jul 24 15:40:55.482265 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.482242 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/whereabouts-cni-bincopy/0.log" Jul 24 15:40:55.507081 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.507056 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fjwh_f7ca19f5-babd-47c0-a92b-be5d34696cd6/whereabouts-cni/0.log" Jul 24 15:40:55.918968 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.918939 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rk7tm_8aadb56a-0ce2-48d2-b3f1-e874409cd867/kube-multus/0.log" Jul 24 15:40:55.984900 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:55.984871 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9v4nd_8213e2c5-8b95-406f-b783-5616c32452a9/network-metrics-daemon/0.log" Jul 24 15:40:56.012104 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:56.012080 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9v4nd_8213e2c5-8b95-406f-b783-5616c32452a9/kube-rbac-proxy/0.log" Jul 24 15:40:56.968392 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:56.968362 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/ovn-controller/0.log" Jul 24 15:40:56.988613 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:56.988591 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/ovn-acl-logging/0.log" Jul 24 15:40:57.012958 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:57.012879 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/kube-rbac-proxy-node/0.log" Jul 24 15:40:57.036868 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:57.036842 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/kube-rbac-proxy-ovn-metrics/0.log" Jul 24 15:40:57.059531 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:57.059508 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/northd/0.log" Jul 24 15:40:57.083088 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:57.083062 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/nbdb/0.log" Jul 24 15:40:57.107458 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:57.107425 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/sbdb/0.log" Jul 24 15:40:57.159553 ip-10-0-133-102 kubenswrapper[2564]: I0724 15:40:57.159522 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ztqv_6755c18e-292e-4b86-b685-41bd508fdcfc/ovnkube-controller/0.log"